Paper short abstract:

This paper examines a critical step in the development of today’s AI systems based on machine learning: the annotation of training data by human experts. Focusing on AI in medical imaging in China, it explores how human expertise gets negotiated, transformed, and inscribed in annotation processes.

Paper long abstract:

This paper examines a critical step in the development of today’s AI systems based on machine learning: the annotation of training data by human experts. With an empirical focus on the application of AI in image-based medical diagnosis in China, the paper will unpack the often laborious yet invisible processes by which human medical expertise gets inscribed and transformed in the annotated medical data used to train AI algorithms. While radiologists specialize in interpreting medical images such as radiographs and writing reports, it has never been a standard, routine practice to label all exact lesions on an image as precisely as machine learning requires. Moreover, such work can oftentimes be contested among medical experts themselves and is extremely time-consuming, especially on a large scale. Drawing on 10 months of ethnographic fieldwork at two Chinese medical AI startups and extensive in-depth interviews with medical image annotators, I will analyze the modes and strategies for medical image annotation as well as the negotiations over credible expertise in the Chinese medical AI industry. In particular, I will highlight the emergence of a nascent profession referred to as “medical annotation specialists” in China, whose work represents a decentralization of expertise in the medical sphere. By opening up the “black box” of AI technology development and examining the human dynamics behind it, the paper will shed light on the co-production of medical AI algorithms and new social orders.