Short abstract

Social media data faces replicability challenges due to restricted access, platform variability, and limited transparency. To address these issues, we propose solutions, such as synthetic data use, development of shared research repositories, and adoption of alternative replication approaches.

Long abstract

Social media data from platforms such as Facebook, X, and TikTok can offer valuable insights into human behavior. It has, hence, become increasingly prominent in research in social and behavioral sciences (but also other scientific fields). However, recent shifts in data access policies—most notably the substantial restriction and monetization of data availability through Application Programming Interfaces (APIs) by platforms such as Facebook and X—have introduced significant barriers to ensuring the reproducibility and replicability of any research based on social media data. This presentation highlights the challenges and complexities of replicating studies with social media data, emphasizing key issues, such as restricted data access, limited data transparency, and temporal/contextual variability of platform content. Drawing on replication attempts in computational social science, we provide an overview of the current state of social media data replications as well as their most common barriers and present empirical evidence on the ephemerality and (non-)replicability of such data. We propose strategies for improving replicability, including an early and incremental preregistration of research, prospective replications, the use of synthetic/intermediate datasets, and detailed and transparent documentation of methods and data sources. We also advocate for collaborations between researchers, the development of shared research material repositories, and the adoption of alternative replication approaches, such as conceptual replications. By addressing these issues, this presentation contributes to a broader conversation on enhancing the reproducibility and replicability of research with social media data, ensuring that research remains robust in the face of a dynamic and volatile online media landscape.