Paper short abstract:

How can (and should) researchers investigate whether a digital health intervention actually works? In this paper, we explore how a digital health approach for self-testing is evaluated and consider how current evaluation indicators make (in)visible the way an app is made to work in practice.

Paper long abstract:

In the case of digital health interventions, the “gold standard” for evaluating effectiveness is the randomized control trial (RCT). Yet, RCT methodology presents issues such as precluding changes to the technology during the study period as well as the use of study settings that do not reflect “real world” contexts. So, how can (and should) researchers investigate whether a digital health intervention works in practice? In this paper, we use insights gathered throughout our ethnographic research on a quasi-randomized trial implementing an app called HIVSmart!, which is digital strategy designed to support people in the process of HIV self-testing. We explore how digital health approaches, such as apps, are evaluated and how current evaluation indicators make (in)visible the daily realities of making an app work in practice. Our analysis reveals that digital health researchers and policy-makers who guide digital health evaluation need to reconceptualize the way we ask questions about digital health effectiveness. For example, instead of asking whether a particular digital health tool for diagnosis/screening is “easy to use”, we should instead take a much closer look at what makes the process of using the technology easy over time and the multiple, shifting users who define this ease of use. Based on our findings, we contribute to critical STS theory on the evaluation of global health interventions by putting forward alternative ways of asking questions about the effectiveness of digital health interventions.