Short abstract

This paper analyzes publication patterns in economics by looking at the number of hypotheses with results reported in studies from the AEA RCT Registry. By tracking hypothesis-level outcomes in over 300 registered trials, it provides unprecedented insight into the size of the file drawer problem.

Long abstract

We developed an approach to standardize and record the hypotheses of studies registered in the AEA RCT Registry. We apply this approach to the registration of 300 studies registered in the AEA Registry between 2015 and 2017. This corresponds to thousands of hypotheses, and we proceed to search for the corresponding results in the published literature, publicly available working papers, and other research reports. This is among the largest follow-up studies in economics aimed at measuring publication bias and selective reporting, and is novel in focusing on results at the research hypothesis level. Preliminary analyses show substantial heterogeneity in the number of registered hypothesis, many of which we were unable to find in the published literature up to nine years later. This result suggests that preregistration alone may not solve the problem of selective reporting. In addition to measuring publication bias, we also develop an innovative approach to standardized reporting of research results in economics – a practice already widely used in other fields, including medicine and public health – that could be of interest to journals, funders, and professional associations.