Paper Short Abstract

Scientists increasingly publish in Open Access (OA) journals. This study compared the statistical quality of psychotherapy research in OA vs. subscription journals. OA journals showed larger samples, higher statistical power, and smaller effect sizes, supporting their role in robust research.

Paper Abstract

Scientists increasingly publish in Open Access (OA) journals, which offer free access to scientific literature and are considered more transparent and rigorous in their editorial and peer-review processes. While Open Science promotes methodological and statistical quality, no study has examined whether this principle is reflected in OA versus subscription-based journals in psychological sciences. This study addresses this gap by comparing the statistical quality of publications assessing the efficacy of psychotherapies for depression in adults.

We retrieved 467 effects from 357 articles in the Metapsy database (Miguel et al., 2022), including 199 effects from 167 OA journal articles and 268 effects from 190 subscription journal articles. We compared observed effect sizes, sample sizes, and statistical power across publication types.

Effects in OA journals exhibited (i) smaller effect sizes, (ii) larger sample sizes, and (iii) higher statistical power compared to subscription journals. Median statistical power was below the recommended 80% threshold for small (d = 0.2) and medium (d = 0.5) effect sizes in both publication types. However, for large effect sizes, OA journal articles had a median statistical power above 80%, while subscription journal articles remained below this threshold.

Our findings suggest that OA journals publish studies with higher statistical quality, likely due to their greater transparency and rigorous standards.