Send message to Author

Paper short abstract

This paper explores the AI-biomedicine intersection, revealing a disparity between scientific discourse and material practice. We highlight how a benchmarking culture prioritises scoring over utility, whilst a long-tail effect centralises focus, fragmenting local biomedical knowledge.

Paper long abstract

This study critically examines the intersection of AI and biomedicine, exploring a structural disparity between scientific discourse and material practice. We analysed a corpus of 44,286 metadata records (from Europe PMC, Crossref, PubMed, arXiv, bioRxiv, and medRxiv) and 1,388 full-text papers (from Europe PMC and PMC) to map the network of biomedical models and datasets. We distinguish between a "MENTIONS" layer—representing discourse—and a "LINKS" layer representing practical, code-level relationships.

Our findings highlight a culture of benchmarking. Within practical links, evaluation-based connections account for 57.2%, heavily outweighing foundational training (40.4%). This suggests a tendency to prioritise performance metrics on standardised datasets, potentially neglecting real-world practical utility. Consequently, models may appear highly effective in metrics but perform poorly in actual application. Furthermore, the network exhibits a long-tail effect: 56.9% of model nodes and 64.8% of dataset nodes function as single-use entities. This reveals a centralised yet fragmented landscape; whilst research concentrates on a few renowned datasets, "local knowledge" risks marginalisation. For instance, rare disease data from local hospitals may struggle to enter mainstream datasets.

Ultimately, this structural disparity points to a potential gap between the technologies broadly discussed in the literature to construct legitimacy or chase trends, and the material operations actively performed by researchers.

Note: The quantitative findings presented herein are based on preliminary tests utilising Qwen-series models. The formal, updated results will be incorporated into the actual conference presentation.