Paper Short Abstract

Some retracted work continues to be cited post-retraction. Some papers cite this work without awareness of its retracted status, but others do (e.g. to attempt to replicate). I present a tool that detects citation intentionality with high accuracy (95%) to help understand if retractions are working.

Paper Abstract

The retraction of unreliable work is an essential part of self-correcting science. However, many papers continue to receive citations after retraction, some even amassing more citations after being retracted than before. While some of these citations come from scholars that are unknowingly relying on discredited research, not all post-retraction citations are a cause for concern. For example, papers may cite a retracted work when they are attempting a replication or pointing to an example of poor or fraudulent science to avoid. Distinguishing between these very different categories of post-retraction citations is an essential step towards understanding how effective retractions are as a corrective measure. Currently there is no way to detect at scale whether papers that cite retracted work are doing so intentionally. Prior work attempting this classification used a rule-based heuristic method that I demonstrate results in many intentional citations being misclassified (high false negative rate). I present an alternative model trained on ~1000 works from the PubMed Open Access Corpus that achieves a 95% classification accuracy rate (F1=0.95) during 10-fold cross validation. Using the more accurate classifier and manual annotation I also present an improved benchmark dataset and discuss how this model can be used to estimate the prevalence of unintentional post-retraction citations in the scientific literature broadly.

*7-1-25: This original abstract reported a 95% accuracy and accuracy score. After cleaning duplicated data that was originally missed, the new pilot results are (10FoldCV to train on 80% of data, test accuracy on held-out 20%) 91.7% and F1 0.91