Paper short abstract:

Many stakeholders regard pursuing reproducibility as essential for enhancing scientific integrity. However, requirements for reproducibility implicate a logic at odds with practices in emerging domains of science. This logic has the potential to harm these domains' long-term prospects.

Paper long abstract:

A growing movement of key scientific stakeholders is promoting reproducibility (the ability to reproduce published scientific findings by replicating the steps in the original analysis) as essential for enhancing scientific integrity. Requirements for reproducibility include use of open source software or code written by researchers themselves, willingness and ability of researchers to share code and data they use, and provision of digital infrastructure for code and data circulation. However, these requirements implicate a "logic of reproducibility" at odds with practices in emerging domains of science, and that could harm these domains' long-term prospects.

This paper presents a five-year study of an emerging domain, subseafloor biosphere research, that involves multidisciplinary studies of seafloor microbial life. Typical of many emerging domains, subseafloor biosphere research is characterized by methodological heterogeneity, limited access to data, disparate approaches to data management, and even disagreement about what is considered "data." Many research workflows are exploratory, drawing together diverse datasets from multiple sources and employing an array of methods. The logic of reproducibility risks impeding domain researchers: digital data infrastructure may encode and impose standards prematurely; researchers often lack the means or rights to share data, or may be unwilling to share data that are scarce and hard-won; and they sometimes have to use proprietary software either because no relevant open source software exists or they cannot write their own code. If scientific integrity comes to entail reproducibility, the credibility and maturation of subseafloor biosphere research could be harmed. Instead, conceptions of integrity should remain pluralistic.