Send message to Authors

Paper short abstract

Taking inspiration from classic STS studies - and arguing critically against current approaches of media studies and critical data studies - the paper outlines a methodology for studying AI governance and society tailored for a study of the Danish government’s taskforce for AI in the public sector.

Paper long abstract

The authors of this paper conduct a 2-year study of ’the digital taskforce for AI’, a group of fourteen civil servant who have been given the task of orchestrating the roll-out of large-scale AI solutions in the Danish Public sector. It is widely expected that the taskforce will play a key role in defining meaningful AI ’solutions’ and meaningful applications in the years to come.

Unsurprisingly, we have faced some problem of access, but the most difficult challenge has been to define a methodological approach that does justice to the phenomenon of a welfare state struggling to make good use of AI amidst geo-political tensions, rapid technological development and public controversies about AI. This tumultuous phenomenon, we argue would be difficult to get into focus if we followed the analytical styles currently developed by STS in dialogue with fields such as critical data studies and media studies. Particularly problematic to our case, we suggest, is the tendency of critical data studies to focus on victims rather than (also) the imperfect attempts to compose common goods. Equally problematic is STS/media studies’ preoccupation with studying semantic patterns across vast media datasets, which in our view jettisons the opportunity of developing stronger historical-narrative explanations of critical decisions and path-dependencies.

The paper presents essential details of our methodological program and its inspiration from classic STS studies of the co-construction of technology and society. We argue that STS has a rich classic toolbox of methodological resources that could and should be mobilized for studying AI.

Send message to Authors

Paper short abstract

This paper explores methodological challenges in studying LLMs through an operational ethnography of GPT. By experimenting with prompts during the sentiment analysis of YouTube comments, the study reveals methodological biases, opacity, and interpretive mediation in AI assisted research.

Paper long abstract

As Large Language Models (LLMs) increasingly shape research practices in the social sciences, they are often approached as efficient tools capable of automating tasks such as text generation, classification, or data analysis. However, this instrumental perspective risks overlooking the complex ways in which these systems participate in and reshape methodological practices.

This paper addresses the methodological challenges of studying contemporary AI systems by proposing an operational ethnography of GPT, both through its chat interface and its API. The study examines the model’s behaviour in a real research setting: the sentiment analysis and visualization of YouTube comments. Rather than treating the model as a neutral analytical instrument, the research focuses on the interaction between researcher, prompts, data, and model outputs.

Through systematic prompt experimentation, the analysis shows that working with LLMs involves continuous iterations, adjustments, and interpretive decisions. Small variations in prompt design generate different classification outcomes, revealing how LLM based analyses are shaped by methodological biases, system opacity, and infrastructural constraints.

Building on debates in Science and Technology Studies and critical AI studies, the paper argues that LLMs should be understood as hybrid methodological entities that simultaneously function as tools, devices, agents, and opaque “pseudo methods.” From this perspective, studying AI requires methodological approaches capable of encircling these systems through experimentation and reflexive engagement. Such approaches make visible the epistemological assumptions and power relations embedded in contemporary AI infrastructures and practices of knowledge production.

Send message to Author

Paper short abstract

This contribution draws on a co-laborative ethnography in a computer science lab in Romania and argues that the stories, or fables (Haraway 2016), told both within the lab and about it are key to understanding algorithmic decisions-making and its inherent societal and political consequences.

Paper long abstract

This contribution draws on an ethnography of a computer science lab in Romania, where software engineers work on the interpretability of image recognition algorithms. It argues that the stories, or fables (Haraway 2016), told both within and about the lab are key to understanding algorithmic decision-making and their inherent societal and political consequences. In the lab, tinkering with deep neural network models, introducing additional layers into the learning process, and creating “visualisations”, such as heat maps, is not only a technical process but also, at every stage, relies on and incorporates storytelling. Computer scientists often use stories and metaphors, through which fabulation is entangled with the material and technical practices of “making”. In addition, fabulation is entangled with the practices of knowledge production about such practices within STS. Through an innovative methodological approach, this paper draws on a collaboration and co-laboration between a social- and computer scientist, and mobilises multimodal methods to argue that different modes of storytelling might enhance our understanding of “black boxed” image recognition algorithms and their societal and political consequences.

Send message to Author

Paper short abstract

How can we study AI futures without reproducing their foreclosures? This paper proposes a methodology of collective autonomy that combines genealogical discourse analysis with speculative and critical data practices to reclaim futuring as a democratic, politically contested practice.

Paper long abstract

Contemporary scholarship on AI faces a distinctive methodological challenge: how to study systems whose opacity, scale, and embeddedness in sociotechnical imaginaries resist conventional analytical approaches. This paper develops a genealogical and discursive methodology for investigating how visions of the digital future are produced, naturalised, and contested — arguing that foregrounding metaphor and discourse renders visible the epistemological assumptions through which post-democratic futures are increasingly normalised.

Drawing on critical discourse analysis and the history of sociotechnical imaginaries, we trace how datafication, smartness, and AI have fostered visions that privilege financial speculation over democratic participation. Attending to metaphor and mediation — drawing on Serres' relational philosophy of interference and entanglement — we encircle AI by mapping the discursive and imaginative conditions of its possibility rather than attempting to penetrate the black box.

We then propose a two-movement methodological intervention. The first, critical and archaeological, reads AI systems against the grain: examining training data, benchmarks, and model documentation as sites where epistemological assumptions and geopolitical hierarchies are sedimented. The second, speculative and anticipatory, uses scenario-building, counterfactual imaginaries, and design probes to enact collective futuring rather than merely analyse it. These practices instantiate methodological collective autonomy: research conducted from within sociotechnical entanglements, oriented toward shared judgment rather than sovereign critique.

Collective autonomy functions not only as a theoretical counter-concept to techno-capitalist futuring, but as a methodological commitment by practising dissent and keeping open the question of what futures remain possible.

Send message to Author

Paper short abstract

The presentation outlines the research design and methodology that was developed for a currently undergoing PhD project on AI governance. The case study is that of a multinational financial company whose activities on AI regulation and use were observed with direct access to the field.

Paper long abstract

The use of AI systems both in the public and private sectors calls for regulation. Recently, many companies have developed and implemented governance models that introduce guidelines and procedures for the use of AI systems. Typically, governance policies adopt a so-called Responsible AI (RAI) approach. The question arises, how to study the forms actually taken by responsibility.

The present work concerns a currently ongoing PhD project that investigates the AI-related activities of a multinational financial company and how responsibility is translated into practices and discourses. Specifically, the project performs qualitative research and was granted access to the field: it was possible to examine company documentation and to interview key actors involved in the design, development and adoption of two AI applications.

The presentation will outline the research design and methodology of the research project. The main challenge concerned addressing the large number of actors involved in the development and adoption of an AI system. Therefore, attention was paid to the boundaries within the company and between the company and external actors such as regulators and consultants. Consequently, the socio-technical networks configuring the two applications under study were reconstructed by following the actors. Both a diary and a weblog were kept, and a template analysis was performed. An interpretive framework to appreciate how RAI is contextually configured was then developed. Alongside preliminary results concerning the effects of AI governance on organizational structure and working practices, the presentation will outline the methodological challenges encountered and the main implications for future research.

Send message to Authors

Paper short abstract

AI systems are rapidly deployed across societal sectors. Yet, the question of what happens after their deployment has remained limitedly studied. In this presentation, we will explore emerging AI maintenance practices to discuss how we can study these often elusive and constantly evolving practices.

Paper long abstract

AI systems are rapidly deployed across societal sectors. Yet, the question of what happens after their deployment has remained limitedly explored; how are these systems maintained, to, for example, avoid ‘model drift’ or ‘performance degradation’ over time? One emerging solution to managing the burden of maintaining AI systems is to deploy additional AI models to maintain existing ones. In this presentation, we will explore this notion through a dual approach of ‘reading computer science papers’ (Amoore et al., 2023) and exploring public and commercial materials that articulate the practices and values of these services for various sectors. With this dual approach, we aim to capture how these emerging maintenance practices are technically and publicly introduced, negotiated, and problematised. Examining technical and public debates in conjunction helps us identify how these two realms inform and reinforce each other, and/or give way to disconnects and frictions between them. Specifically, we ask what forms of value ‘AI to maintain AI’ promises to bring and to whom, and how the act of ‘maintenance’ is being reconfigured in these practices. In doing so, we bring STS scholarship on maintenance and repair into contact with Critical AI scholarship studying the value production in AI economies. With this presentation, we aim to use our ongoing research on AI maintenance practices and the methodological struggles we encountered to discuss how we can study often elusive and constantly evolving emerging AI practices in ways that allow us to question the economic and epistemological premises that underpin them.

Send message to Authors

Paper short abstract

This contribution focuses on methodological challenges of researching AI with digital methods in a platform context. Structured along the lines of material, temporal, and epistemological access, it demonstrates both the limitations and potentiality of digital methods in relation to AI's complexities

Paper long abstract

There have been many impulses to make sense of various functionalities under the ‘artificial intelligence’ umbrella, with diverse methodologies employed. Such methods often strive to perform AI critique through a cross-disciplinary lens to historicize, denormalize, and engage with the present moment of an apparent ‘inevitability’ and omnipresence of AI. The practice of operationalizing digital methods, in the spirit of ‘mapping’ AI, is one such methodology. However, how does AI introduce and deepen (not-so-new) messiness in doing digital research? What challenges do digital methods carry when we apply them to today's digital landscape ploughed with and about AI? And, crucially, what limitations do we face when we use digital methods to make sense of AI? This contribution focuses on three methodological challenges in researching AI with digital methods in a platform context. To do so, this methodological contribution is organized along the three problematizing axes of the material, the temporal, and the epistemological access.

Inspired by the traditions of STS and critical AI studies, as well as media studies, this methodological proposition discusses the technical, political, and epistemological limitations and challenges of researching AI with the digital, and puts forward a layered approach to thinking about methodological and analytical entry points to AI critique. Ultimately, it demonstrates how a critical recognition and engagement with the limitations and challenges we face in researching AI with digital methods can serve as an empowering counter-position to the hegemonic claims of AI’s completeness, frictionlessness, and universality.

Send message to Authors

Paper short abstract

NSFW (not-safe-for-work) data has repeatedly been flagged in ML training datasets. Instead of addressing this as a dataset issue, we discuss a mixed methods approach tracing NSFW platform taxonomies across dataset acquisition, model training, and system outputs while reflecting on ethical tensions.

Paper long abstract

Despite filtering efforts, not-safe-for-work (NSFW) data has recurrently been identified in large-scale multimodal training datasets, largely due to web-scraping (Birhane et al. 2021; Thiel 2023). Such content is commonly framed as a dataset problem focused on explicit content detection. With the development of foundation models, emergent video AI systems, and decreasing insistence on guardrails, the societal implications of NSFW data have become a matter of public concern (Birhane et al. 2024).

Existing approaches focused on dataset cleaning are insufficient, as treating NSFW content solely as a dataset issue overlooks the broader taxonomies scraped directly from NSFW platforms. These structure subjects through intersectional racialised and sexualised classifications, which shape downstream ML systems. Platforms thus function as large-scale annotation infrastructures whose logics move beyond their original context. To attend to these sidelined dimensions of NSFW data, we discuss a pipeline approach tracing NSFW content across platforms, datasets, ML processes, and model outputs. This requires a mixed methods approach, from quantitative platform taxonomy analysis to investigations in development practice and qualitative interpretation of system outputs.

In studying the more opaque stages, such as dataset curation and model training, access is usually restricted. This introduces ethical tensions, as researchers may need to ally with projects whose goals they do not embrace to acquire necessary insider information. Furthermore, sustained exposure to explicit content and taxonomies can impact researchers' well-being. Addressing these tensions requires reflexive methodological practices to situate the auditing of NSFW infrastructures in broader debates on research ethics and data labour.

Send message to Author

Paper short abstract

This paper reflects on five AI methods used by the author over the past five years. Together, the paper considers the success of deploying them in different research and pedagogic settings, and the value of using them to test and critique an array of AI phenomena from outputs to supply-chains.

Paper long abstract

This paper is a reflection on five AI methods used by the author in different scenarios over the past five years, from empirical work studying AI competitions and autonomous driving, to postgraduate seminars studying AI controversies and PhD workshops exploring creative engagement with generative AI tools. Together, the paper details five AI methods I refer to as: creative AI methods, LLMs as topic modeller, AI technography, operational methods, and public AI repositories. Collectively, the approaches contribute to innovative methodological work in STS, media studies, computer science, and associated fields that have, in recent times, been developed to use, test, and critique the huge array of AI tools that have made their way into the public and academic consciousness since 2022, from the slick user interfaces of genAI products like ChatGPT to eminently usable topic modelling LLMs like BERTopic. Drawing on academic traditions and trajectories that have used AI as a method or methodological approach, I argue that each method constructs a specific version, and vision, of AI. I understand these different versions as constructing AI as: user interface, automated tool, innovation practice, developmental process, and public controversy, respectively. I reflect on the success of testing them in research and pedagogic settings, and the relative value of using such approaches for critiquing – amongst other things – AI outputs, discourses, supply-chains, and effects. The paper concludes by offering suggestions for how AI methods may be further developed to aid the use, testing, and critique of AI phenomena in the future.

Send message to Authors

Paper short abstract

This paper proposes mechanology and transduction as methods for studying how ML operates, rather than only what it does. Drawing on ethnography in an AI lab, it shows how computational processes enact regimes of knowledge, calling for new methodological engagements with algorithmic opacity.

Paper long abstract

This paper reflects on methodological impasses encountered during ethnographic research in an AI laboratory, and engaging with ‘technology’ more broadly. Conventional approaches that privilege either engineers’ interpretations of their systems, or the social effects of algorithmic outputs, often leave the technical operations opaque to anthropological inquiry. We argue that these limits call for methods that engage not only with what machines do, or what practitioners say they do, but with how they operate, reorganising modes of perception, action, and knowledge.

Drawing on debates starting from the philosophy of Simondon, this paper develops two methodological orientations: mechanology (Rieder, 2020) and transduction (Mackenzie, 2002; Helmreich, 2007). First, we propose a mechanological reading of encounters with ML systems, as a way of translating machinic operations into anthropological problems, tracing how computational processes crystallise regimes of generativity and sociality. Second, we outline what a transductive anthropology might look like: one concerned with how signals traverse technical media and, in doing so, do not merely pass through, but transform the very terms they relate, reshaping concepts and practices by constantly redrawing the limits of thinking and being, in ways our interlocutors often refused to acknowledge.

These approaches foreground algorithmic techniques as epistemic actors whose modes of functioning demand new forms of methodological engagement. Rather than treating opacity as a barrier to critique, this paper argues for studying algorithmic operations as sites where epistemological assumptions and social imaginaries are materially enacted, opening pathways for critique that do not rely on rendering these systems fully visible.

Send message to Authors

Paper short abstract

This paper challenges standardized AI evaluations by combining STS/Critical AI Studies methodologies with methods from red teaming and adversarial testing to illustrate how contrarian thinking, hands-on tinkering, and speculative reasoning can (re)configure AI-oriented methodological agendas.

Paper long abstract

In the field of AI research and development, the use of quantitative AI benchmarks constitutes a key means through which the capabilities and risks of AI models and systems are evaluated and “known”. Quantitative AI benchmarks come in many shapes and forms, ranging from multiple-choice questionnaires to frameworks for assessing the free-text outputs of AI models, and evaluations more akin to psychological intelligence tests. As of late, however, the types of knowledges gained through AI benchmarking has been heavily questioned, leading scholars and to speak of an ongoing “evaluation crisis” in the field of AI. This paper sets out to problematize quantitative AI benchmarks and benchmarking practices, asking how humanities scholarship and qualitative methods can challenge what counts as “true” and “meaningful” insights about generative AI models and systems. In particular, it explores how methodologies developed in STS and critical AI studies – combined with methods borrowed from the field of red teaming and adversarial testing – can challenge quantitative, standardized, and metrics-driven forms of AI knowledge production. It also reflects on how red teaming and adversarial testing – as first formalized by the U.S. Military during the Cold War and later developed in fields like Cybersecurity – can foster inventive methodological interventions that welcome ambiguity and multiplicities of meaning while problematizing epistemological assumptions about AI. Doing so, it illustrates how adversarial tactics, contrarian thinking, hands-on tinkering, and speculative reasoning can provide fruitful ground for (re)configuring AI-oriented methodological agendas.

Send message to Authors

Paper short abstract

We propose to investigate how models take shape through what we call “learning work”: the modelling practices through which AI systems are continuously updated to keep pace with shifting sociotechnical environments.

Paper long abstract

Ethnographic research into developers’ modelling practices constitutes an emergent field of investigation. Methodological strategies to foreground this often-hidden labor become necessary to unveil how models take shape. To address this need, we suggest considering two moments as methodological entry points. First, transfer learning, that is, routine adaptation of general-purpose pre-trained models to specific requirements. By examining what developers (out of necessity) choose not to modify, alongside the minute technical features they do alter during such model tailoring, an infrastructural map of how things are tied together can be drawn. Second, the detection of shortcut learning, i.e., developers’ struggle to discern when a model is diverging from its expected path and to identify the specific patterns it has prioritized to avoid embedding these diversions in subsequent upgrades. As developers catch the model’s spurious correlations, the underlying rules of both the model and the working team’s decision-making become visible. These two methodological entry points are instances of what we propose to call “learning work”: the updating of AI systems in the face of ever-changing, present-tense contingencies. Legacy patterns continuously turn out to be stale, thus learning practices become essential to keep these systems alive, meanwhile revealing the disciplinary stakes of what is apprehended and what is excluded from the system’s reality.

Send message to Author

Paper short abstract

This paper presents a responsive methodology for studying medical AI deployment as an emergent, contested process. Drawing on multi-study research, it reflects on the epistemological, political-economic, and temporal challenges of tracking shifting objects across a turbulent and heterogeneous arena.

Paper long abstract

Studying medical AI deployment poses distinctive methodological challenges. The object of inquiry is not a stable technology but an ongoing accomplishment, continually reassembled by vendors, healthcare institutions, intermediaries, and regulators. Strategies shift mid-research, problem definitions mutate, and the boundaries of "the system" are themselves contested. Traditional STS approaches struggle to capture this moving target, while established framings of platformisation risk importing assumptions of planful disruption that obscure the dispersed, contingent character of what actors actually do.

This paper presents a reflexive, responsive design developed across a multi-study thesis on medical AI deployment. Rather than fixing the research object in advance, the approach adapts iteratively to practitioners' shifting understandings, combining landscape mapping, organisational vignettes, and longitudinal case study to trace deployment as continued formation work, a form of methodological "encircling" that follows how actors borrow, imitate, and mobilise strategies across a heterogeneous arena.

Empirically, the approach reveals dynamics fixed-frame methods would miss: how platformisation becomes multiple as actors enact divergent ontologies, and how a hospital created a "platform of platforms" resisting vendor dependency, contrary to dominant platform narratives. These findings emerged through sustained responsiveness to a turbulent ecology, not predetermined design.

The paper reflects on three entanglements: studying objects whose boundaries are actively contested; navigating layered opacities across vendors, procurement, and regulation; and registering how imaginaries and materialities co-evolve amid hype and volatility.

Send message to Authors

Paper short abstract

This paper presents a research protocol to study how workers use LLMs in practice. Through a six-month ethnographic studio inquiry with 32 participants, we developed a method for observing LLMs-in-use and identifying their consequences for work, challenging deterministic accounts of AI’s impact.

Paper long abstract

Three years after the introduction of ChatGPT, around 40% of people in the US and Europe use Large Language Models (LLMs), yet we still know little about what it means to “use” LLMs at work. Much of the current discussion focuses on predicting the impact of AI or extrapolates from the properties of LLMs (e.g., bias or confabulation) instead of studying how they are enacted in ordinary practices. Our paper presents the protocol we implemented to examine what users value in these systems and what they construe as problems.

We revisit the ethnographic tradition of studying technologies-in-use (Suchman 1999; Orlikowski 2000) to address the methodological challenges of GenAI. Observing LLM use is difficult because interactions are individualized, locked behind proprietary interfaces, and embedded in a normatively saturated environment that encourages workers to report what they should do rather than what they actually do. Describing these systems also poses difficulties for STS scholars, as their probabilistic and task-agnostic nature challenges classical notions such as the “script” (Akrich 1992).

To address these challenges, we designed a research “studio” in which workers participated as co-inquirers. Using a structured exercise protocol spanning six months, participants brought conversation logs, described their practices, and collectively reflected on concrete situations. Four cohorts of workers (32 in total) followed this protocol. This protocol allowed us to expand our analytical repertoire and to identify several consequences of LLMs-in-use. It also helps deflate deterministic narratives about AI’s impact on work while decentering technologists’ perspectives by foregrounding workers’ experiences.

Send message to Authors

Paper short abstract

The paper proposes “hypothetical enrolment” as a methodological contribution to move between the expectations about AI diagnostics and their integration into clinical settings. This approach allows to appreciate the potential organizational and epistemic consequences of adopting AI diagnostics.

Paper long abstract

Despite the supposed potentialities of AI tools for medical diagnosis, their adoption is a slow and troubled process. Recent empirical studies illustrated the misalignment between the narratives and expectations about these tools and how they work in real-world settings (Carboni et al. 2023, Kusta et al. 2024). These studies suggest that the adoption of AI diagnostic tools transforms the organizational workflows, professional competences and epistemic practices in the clinical settings in which they are deployed. Yet, there is a substantial lack of research frameworks and methods for addressing the prospects of integrating AI diagnostic tools into real-world settings (Williams et al. 2024).

To assess the potential consequences of adopting AI tools, we propose “hypothetical enrolment” as a methodological framework. We conceive of “hypothetical enrolment” as a situated, anticipatory and performative approach. Our methods focuses on actors’ expectations about the consequences brought by intelligent machines in clinical practices and it analyzes such expectations by paying attention to the knowledge infrastructures in which AI tools would be implemented. We tested the validity of our method against an empirical case, the start-up Autism Scope (AS). AS applies machine learning models for the early detection of autism-spectrum-disorder. We conducted interviews with AS developers and with neuropsychiatrists, exploring the “hypothetical enrolment” of AS in clinical settings. Overall, our method offers a methodological contribution which combines two interrelated dimensions: first, actors’ promises and expectations about AI diagnostic tools; second, the infrastructural and organizational features shaping the settings in which they would be adopted.