Short abstract

We replicate earlier studies on the Matthew effect and the early-career setback effect across seven research funders worldwide and study their generalisability. Initial results show some support for the Matthew effect, but cast some doubt on the early-career setback.

Long abstract

Previous literature showed that researchers who were awarded early-career funding (1) were more likely to be awarded later-career funding (the Matthew effect); and (2) showed a lower impact than those who got rejected but continued in science (the early-career setback). We replicate both studies across seven research funders worldwide (Canada, UK, Denmark, Luxembourg and Austria). The data we collected covers over 80,000 funding applications and over a dozen different funding programmes.

The original studies rely on a regression discontinuity design. In most funding programmes we studied, there is no hard cutoff based on review scores, therefore making it difficult to implement a regression discontinuity design. Instead, we implement a fuzzy regression discontinuity design. The regression discontinuity design focuses on the local effects around the funding threshold only. We aim to study the generalisability of the effects based on a hierarchical Bayesian model using latent variables.

Preliminary analyses for three funders show mixed results. One funder shows an unexpected negative Matthew effect, but other funders show positive effects, as expected, but the differences are not significant. The early-career setback is visible in two funders, but differences are again not significant. Results from the Bayesian model for one funder suggest that the Matthew effect generalises, but the early-career setback does not. The methodology still needs to be finalised, so results are potentially subject to change.

We expect to present full results across all funders, including a full sensitivity analysis of the Bayesian model during the conference.