Assessing Reproducibility in Economics: Using Standardized Crowd-sourced Analysis
Seung Yong Sung
(UC Berkeley)
Lars Vilhuber
(Cornell University)
Edward Miguel
(University of California,Berkeley)
Fernando Hoces De La Guardia Figueroa
(UC Berkeley)
Abel Brodeur
Short abstract
We will present the Social Science Reproduction Platform (SSRP), a tool for conducting and logging standardized computational reproduction exercises. Drawing on 400+ reproductions in economics, we find that 30–38% of recent studies meet at least a basic standard of reproducibility.
Long abstract
We will present the Social Science Reproduction Platform (SSRP), a platform for publicly recording computational reproductions in economics and other social sciences using a standardized framework. SSRP enables crowd-sourced evaluations of computational reproducibility at the table or figure level a 10-point reproducibility scale, and aggregates these assessments to the claim and paper level. Drawing on over 480 reproduction exercises—including over 410 in economics—we document substantial heterogeneity in reproducibility scores. Approximately 30 to 38% of recent economics papers reproduced on SSRP achieve some form of computational reproducibility (from either raw or analysis data), though only a subset is fully reproducible from raw data.
Accepted Paper
Short abstract
Long abstract
We will present the Social Science Reproduction Platform (SSRP), a platform for publicly recording computational reproductions in economics and other social sciences using a standardized framework. SSRP enables crowd-sourced evaluations of computational reproducibility at the table or figure level a 10-point reproducibility scale, and aggregates these assessments to the claim and paper level. Drawing on over 480 reproduction exercises—including over 410 in economics—we document substantial heterogeneity in reproducibility scores. Approximately 30 to 38% of recent economics papers reproduced on SSRP achieve some form of computational reproducibility (from either raw or analysis data), though only a subset is fully reproducible from raw data.
Meta-economics? Meta-research innovations in economics
Session 1 Monday 30 June, 2025, -