Paper Short Abstract

From 1978 till end of 2024, 67 studies analysed differences between preprinted or submitted versions vs peer reviewed journal versions. Our meta-analyses show that while some aspects change, conclusions rarely do (in 2% of cases). We are launching a living review website to showcase all results.

Paper Abstract

Previous research has indicated a knowledge gap on changes brought on by peer review and journal publishing. We are conducting a living evidence synthesis of studies that analysed differences between preprinted or submitted versions vs peer reviewed journal versions. We identified 67 studies published from 1978 till the end of 2024, of which 33 (49%) analysed changes between preprint and journal versions, 26 (39%) between submitted and published versions, and 10 (15%) between rejected versions and those later published in other journals. All but 3 studies looked at different sets of outcomes. Furthermore, 45 (67%) analysed changes manually, 14 (21%) used computational methods, and 8 combined the two (12%). The median number of analysed version-pairs was 109 (IQR 48 to 388). Narrative synthesis indicated that studies reported high similarity between version-pairs, with meta-analyses showing highest frequency of changes in title, authorship, conflicts of interest, and numerical results, with rare impact on study conclusions (i.e. 2% of analyzed pairs, 95%CI 1 to 4). However, with almost 7 million articles published per year, the combined sample size covering almost half a century of research on this topic is likely inadequate for proper results generalizability, especially as no data points exist for many fields. We have therefore approached publishers and editors to greatly increase data availability that would allow larger samples to be compared. We are also launching a live tracker website that we plan to present at the conference.