Paper Short Abstract

We evaluated eleven confirmatory multi-laboratory projects to assess preclinical research robustness. Our analysis revealed enhanced rigor as the project progressed from exploratory to confirmatory stage. We examined effect sizes and significance to compare single- and multi-lab studies.

Paper Abstract

Many promising preclinical research findings fail to translate into clinical practice, as numerous interventions ultimately fail during the costly clinical trial phase. One proposed strategy to improve evidence generation along the preclinical trajectory is introducing multi-laboratory (multi-lab) confirmatory studies with increasd rigor and transparency. To investigate their potential benefit, we assessed eleven projects that conducted confirmatory research in a rigorous multi-lab set-up. We accompanied the projects during their four-year funding period and provided methodological support. Through this, we have generated a unique database that allowed us to analyze protocols and primary data from exploratory phase, pre-registration, and confirmatory stage. We evaluated the reliability and validity of the projects through a guided self-assessment of robustness, aiming to identify potential improvements from the exploratory to the confirmatory phase. Overall, most projects showed enhanced research rigor over time. Internal validity improved through the implementation of blinding and randomization, while external validity was strengthened by incorporating diverse disease models, including both sexes, and implementing a multi-lab study design. However, projects also reported set-backs within the multi-lab setup, for example differences in animal welfare regulations resulting in severe delays. We conducted a meta-analysis of each project's confirmation success, evaluating effect sizes and statistical significance. In conclusion, our analysis of a convenience sample offers a nuanced perspective on the translational relevance and potential pitfalls of preclinical confirmatory multi-laboratory studies.