Our aim is to catalyze new experimental collaborations between researchers, funders, publishers and institutions interested in generating robust evidence on how to improve research. Showcasing existing experiments and matchmaking around new proposal development. Let’s get experimental!
Long Abstract
Organised in partnership with Open Philanthropy and RoRI’s AFiRE programme, these three linked sessions will facilitate matchmaking and networking for experimentation across all areas of metascience, with a focus on interventions to support higher quality, lower cost and more impactful research.
Each session will showcase metascience principles, methods or examples of experimentation, as well as providing a platform for co-developing new project ideas by participants. Researchers, funders, universities, publishers and other actors in the research ecosystem are invited to propose experiments and matchmake with potential collaborators.
The Abundance and Growth Fund at Open Philanthropy is happy to consider proposals that emerge from this process, with an aim to fund pilot work on the most promising opportunities for generating credible evidence to support improvements to the research system. Projects with a strong analysis strategy, whether experimental, quasi-experimental or exploratory, will be favoured. Facilitation will be provided by RoRI’s AFIRE programme. Let’s get experimental!
This session will offer an introduction to the use of experimental approaches in metascience, giving participants the opportunity to reflect on how they could set up experiments within their own organisations to improve the design of its programmes and activities.
Long abstract
There is growing recognition of the role that experimentation can play to identify the most effective approaches to accelerate scientific progress, yet adoption remains very low.
This session will offer an introduction to the use of experimental approaches in metascience, drawing on the Experimental Research Funder’s Handbook and the Accelerator for Innovation and Research Experiments (AFIRE), a joint collaboration between RoRI and IGL.
Meta-research ethics demand nuanced governance; what is acceptable and what is not? Inviting diverse perspectives, this panel explores varied meta-research contexts, from academic studies to operational audits, with complex considerations of methods, risks, and comparative research standards.
Long abstract
Should meta-research be conducted under the same ethics and governance frameworks as other research? The obvious is answer is, yes, but should the same rules apply in all cases? What about trialling narrative CVs versus traditional CVs in promotion and hiring practices? Or A/B testing of two ways of handling export control compliance within a university? Or comparative studies of the performance of large language models and human peer reviewers for major conferences...?
Clear frameworks for the ethical conduct of research exist in certain research domains, such as biomedical research, but even here some landmark principles are not well-observed (such as the requirement for study registration in the Declaration of Helsinki ever since the 2013 revision – article 35).
The broad church of meta-research covers a spectrum from publishable academic studies to operational research and audit. Different kinds of investigators and participants are involved across this spectrum. Arguably different protections will be necessary in different contexts, and choosing which apply, and when, requires some fine judgements. Factors might include the research methods used, and who holds the risks, and – for comparative studies – what the unit of randomisation is (if randomisation is used) and whether equipoise is possible or ethical.
This panel will set out various perspectives on these issues, and invite the session participants to contribute. Notes will be taken and an open access report, under the Chatham House rule, will be published.
Tom Stafford
Misha Teplitskiy (University of Michigan)
Short Abstract
Our aim is to catalyze new experimental collaborations between researchers, funders, publishers and institutions interested in generating robust evidence on how to improve research. Showcasing existing experiments and matchmaking around new proposal development. Let’s get experimental!
Long Abstract
Organised in partnership with Open Philanthropy and RoRI’s AFiRE programme, these three linked sessions will facilitate matchmaking and networking for experimentation across all areas of metascience, with a focus on interventions to support higher quality, lower cost and more impactful research.
Each session will showcase metascience principles, methods or examples of experimentation, as well as providing a platform for co-developing new project ideas by participants. Researchers, funders, universities, publishers and other actors in the research ecosystem are invited to propose experiments and matchmake with potential collaborators.
The Abundance and Growth Fund at Open Philanthropy is happy to consider proposals that emerge from this process, with an aim to fund pilot work on the most promising opportunities for generating credible evidence to support improvements to the research system. Projects with a strong analysis strategy, whether experimental, quasi-experimental or exploratory, will be favoured. Facilitation will be provided by RoRI’s AFIRE programme. Let’s get experimental!
Accepted papers
Session 1 Monday 30 June, 2025, -