Log in to star items.
- Convenors:
-
Dipanjan Saha
(University of Liverpool)
Jakub Mlynar (HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland)
Send message to Convenors
- Format:
- Traditional Open Panel
Short Abstract
This panel seeks empirical studies on how diverse users of AI practically reason with, evaluate, and manage its messy, unpredictable outputs, moving beyond the hype of 'disruption' to critically assess its real-world limits.
Description
The inflationary claims of ‘disruption’ surrounding the ‘new’ Artificial Intelligence (AI), often propagated by the corporations developing them, hinder our understanding of both the true potentials of these technologies and challenges related to their integration in social life. To get a better grasp on their rapid advancements and their future consequences for various types of jobs, we need to understand how AI is made relevant to its specific contexts of use. While they are publicly presented as seamless or autonomous tools, such technologies are often messy, unpredictable, and prone to generating outputs that users find ambiguous, problematic, or simply incorrect. This creates a critical gap between the AI's imagined or prescribed use and the practical, situated work required for its smooth operation. Attending to the broad sphere of activities that takes place to make AI work can provide a more measured and empirically grounded basis for evaluating its achievements and limitations as part of its entanglements with our everyday lives.
This panel invites empirical investigations that uncover the lived difficulties of working with various applications of AI. We welcome contributions exploring, but not limited to, the following questions:
• How do a wide range of users, from domain experts to laypersons, actually manage and make sense of the results produced by AI technologies in practice?
• What mundane methods and practical reasoning skills do people employ to evaluate, trust, or challenge AI’s outputs?
• How can we empirically study the multiplicity of reasoning styles and ad-hoc procedures users adopt when evaluating AI-produced results?
• What does attending to these practical difficulties reveal about the actual, rather than promised, capabilities of automation and the necessity of situated human competences?
By focusing on the ‘just how’ of AI’s use, this panel will contribute to the ongoing debates surrounding the future of work, human–machine collaboration, and the observable societal implications of ‘disruptive’ technologies.
Accepted papers
Session 1Paper short abstract
This paper examines how users of AI-driven advertising systems engage in practical reasoning to manage opaque, unstable, and unpredictable algorithmic outputs. Drawing on ethnographic research, it shows how accountability and trust are enacted without transparency in everyday human–AI interaction.
Paper long abstract
AI-driven advertising platforms are widely promoted as autonomous, efficient, and self-optimizing systems. In everyday use, however, these systems often generate unstable, ambiguous, or unexpected outcomes that require continuous human interpretation and intervention. This paper examines how users of AI systems engage in practical reasoning to manage such persistent algorithmic disruption in real-world settings.
Drawing on 27 in-depth interviews, ethnographic immersion, and a netnographic analysis of over 1,000 professional forum discussions (July 2024–July 2025), the study focuses on Pay-Per-Click (PPC) specialists working with AI-based tools such as Smart Bidding and Performance Max. Rather than exercising direct control over algorithmic processes, practitioners develop situated forms of reasoning that allow them to assess, interpret, and respond to fluctuating system outputs. These practices include shared heuristics, collective sensemaking, and affective attunement to campaign instability—often described as “feeling” when something is wrong before anomalies become visible in performance metrics.
I argue that algorithmic disruption is not a temporary condition but a durable feature of AI-mediated work that reorganizes how knowledge, trust, and responsibility are practically accomplished. I introduce the concept of algorithmic responsibility to capture how accountability is enacted through everyday interpretative practices in the absence of transparency or explainability. By foregrounding how AI is reasoned with in practice, the paper contributes to STS debates on human–AI collaboration, the limits of automation, and the situated competencies required to make ostensibly autonomous systems function in everyday life.
Paper short abstract
Based on multimodal analysis of 87 autonomous vehicle left-turn maneuvers recorded by end users as beta testers, this study shows how safety is assessed in real traffic. Rather than inherent in technology, safety emerges as a graded evaluation through practical reasoning and human coordination.
Paper long abstract
Safety is not an inherent property of technology but is produced in situated interaction. This study examines how the safety of autonomous vehicles (AVs) is constituted and assessed by beta testers taking AVs on city rides. Drawing on multimodal conversation analysis (Goodwin, 2000; Mondada, 2014) and a collection of 87 left-turn maneuvers, a hazardous and complex scenario (Choi, 2010), the study analyzes YouTube videos recorded from multiple camera angles by testers themselves.
The findings, based on tests involving at least two end-users with access to Tesla’s autonomous beta software, show that maneuvers are collaboratively evaluated as unsafe, “not safe” or “not unsafe”, revealing a graduated spectrum of safety rather than a binary distinction. The analysis shows how evaluators’ practical reasoning (Garfinkel, 1967) organizes these judgments by attending to the traffic context, the situated spatial and temporal unfolding of vehicle operation, and the projected actions of other road users. In doing so, human actors coordinate with autonomous systems to manage risk and stabilize the technology within a situated sociotechnical framework (Raudaskoski, 2023; Due, 2024).
Although autonomous systems are often framed by media, industry, and some scholarly literature as “disruptive” (e.g., Brynjolfsson & McAfee, 2014; Ford, 2015), real-world performance depends on human judgment and situated action (Suchman, 2007). By foregrounding the “just how” of human-machine interaction, this study highlights the invisible labour and practical accomplishments through which humans make AV systems function reliably, challenging inflated narratives of technology disruption.
Paper short abstract
Office workers learn LLM prompting not as a discrete skill but as a relational everyday practice, which complicates promised productivity gains. Their interactional work and practical reasoning diverge from official AI programs shaped by hierarchy, time, and access to AI tools and policies.
Paper long abstract
Drawing on in-depth interviews with employees of a large organization, this paper examines how workers learn to prompt LLMs and evaluate model outputs, attending to the gap between organizational prescriptions and workers’ practice.
The organization has invested in AI adoption infrastructures: internal platforms, usage guidelines, and teams tasked with spreading prompting knowledge. Prompting is institutionally framed as a learnable, discreet skill to be acquired through sanctioned channels (Korzyński et al. 2023; Liu et al. 2023). Yet many workers are unaware of these resources, do not use them or find them too generic.
Instead, workers construct their prompting competence relationally through peer exchange with colleagues whose domain expertise is situationally relevant, experimentation, and LLMs themselves as interlocutors for refining prompts (Mahdavi Goloujeh et al. 2024; Pakarinen & Huising 2025). Access to the time required is unevenly distributed. The widespread use of ChatGPT instead of the endorsed Copilot as “shadow AI” (Mansner 2025) further exemplifies the distance between prescribed and situated learning practices (Holton & Boyd 2021; Oudshoorn & Pinch 2003).
When evaluating outputs, workers rely on experience and repetition rather than on formal criteria: judgements draw on “experience from past outputs” and “a bit of feeling.” Prompts are rewritten iteratively when results fall short, and what works is discovered by experiment (Zamfirescu-Pereira et al. 2023). We argue that the practical reasoning involved in “learning how to prompt” is a relational accomplishment, shaped by organizational structures yet irreducible to them (Eyal & Pok 2011; Eyal, 2019).
Paper short abstract
After examining a recurring practice among blind individuals—using the sound of tapping on objects as both a summons and a spatial resource for co-present blind participants—we explore the situated relevance for multimodal voice agents of perceiving and responding to more than vocalized sounds.
Paper long abstract
Although paralinguistic features like pitch and speed are being explored in research on voice-based conversational agents, the broader soundscape of a spoken interaction is so far not taken into account by these devices as they generate their responses. Yet, human talk-in-interaction commonly indexes co-occurring non-vocalized sounds—such as the rumble of traffic, or the revealing noise an object makes when struck with one’s hand. We illustrate this emergent relevance of non-vocalized sounds by detailing a routinized practice produced by blind individuals: tapping on objects both as a summons and as a resource available to co-present blind participants in locating the object segmented and made relevant by this tapping. Then, turning to episodes of interaction between blind participants and multimodal voice agents, we examine how the design of these agents prevents users from efficiently mobilizing this repertoire of “tapping” methods they originally developed from and for human interaction. We show how users remedy this difficulty in securing a common referent through a step-by-step upgrading of their indexical practices. Building on this analysis, we shed light on a philosophy of language embedded in the data streams conversational agents are designed to process and respond to. Specifically, we argue that these agents enact a definition of language as stemming exclusively from the human vocal apparatus—and, for most current voice agents, as articulated speech.
Paper short abstract
We use discourse analysis to analyse a 2-day 'chat' with Copilot in which we instructed it to perform an relatively simple teaching admin task, but which led to many difficulties before it was done. Our data analysis highlights its and our rhetorical strategies in identifying and solving the problem
Paper long abstract
The main intellectual contribution we make is to apply established approaches from conversation and discourse analysis to explore the sequence of discursive moves in an extended 'chat' between Co-Pilot and the three authors (all of whom teach the 'dissertation' module of the MA Education and Technology at UCL). We aim to build on existing research into team coordination and cooperation in workplaces, but extend this to the analysis of AI as an additional, new agent in the performance of work tasks.
The data we draw on are the transcripts of our 2-day chat with Co-pilot in February 2026. It was not generated as research data: we did not set out to research discursive positioning and task negotiation in the operation of a teaching task, but to use AI in the administration of the course's dissertation module. However, this proved more challenging than we expected, for several reasons: CoPilot produced unusable results but could not explain why; Copilot's capabilities varied unexpectedly depending on the licensing conditions of the account we used; we made assumptions about the semantic coherence of the data we gave it, because it was coherent relative to our teaching aims. We will present the main points of our detailed linguistic analysis to explore how we re-defined the task repeatedly to make it manageable for Co-Pilot, and also how Co-pilot maintained 'face' and communicated the complexity of its labour to us as its users. Our analysis shows the work required to make AI useable in higher education teaching.
Paper short abstract
This paper examines recent pleas for a “critical ethnomethodology” in light of a reverse-engineered and replicated proto “AI” system, namely Calvin Mooers’ Zator® machine, developed at MIT in the 1940s and 1950s. Thereby, the paper outlines a performative critique of quotidian “AI” labelling today.
Paper long abstract
In Staying with the Trouble, Donna Haraway (2016) highlights two problems associated with “technological determinism” (on its curious persistence, see Wyatt 2008). The first problem concerns a “comic faith in technofixes” (2016:3-4). The second problem, more serious, finds its routine expression in a “game-over attitude” with respect to alternative scenarios (ibid.), effectively “locking in” the future in the name of one technology, system, or process. In response to Haraway’s twofold concern, this paper examines recent pleas for a “critical ethnomethodology” (e.g., Scheffer 2021) in light of a reverse-engineered proto “AI” system, namely Calvin Mooers’ (1951) Zator® machine, a mechanical information retrieval device, developed at MIT in the 1940s and 1950s (Ceruzzi 2019). What did we learn from reverse engineering and, in fact, rebuilding a physical replica of that “low tech” machine, including its “coding scheme for edge-notched cards” (ibid., p. 69)? To address this question, the paper first revisits Garfinkel’s and his students’ texts (e.g., Wood 1969) on Mooers’ “Zatocoding” and “[library] catalogs” as providing ethnomethodological alternates to formal accounts of “practical action and practical reason” (Garfinkel 2002:128). Second, the paper brings to bear its technical insights from machine rebuilding, and multiple keyword searches in a single database, on critical inquiry into current uses, if not corporate abuses of “AI” (e.g., friendliness scoring at Hamburger places). In short, our mid-20th century detour contributes to a performative critique of quotidian “AI” labelling today, as a provocative gesture in (digital) STS (Woolgar 2004; Vertesi & Ribes 2019), if not “critical ethnomethodology” indeed.
Paper short abstract
This paper ethnographically shows how Chinese food delivery platform workers manage unreliable and messy AI-enhanced dispatching outputs and keep platform logistics running through everyday collaborations underpinned by extra immaterial labour.
Paper long abstract
Mainstream research on location-based platform work often places AI-enhanced algorithmic control at the centre of analysis, assuming its efficiency and effectiveness and lacking attention to the everyday practices required to keep these systems running smoothly. In practice, however, platform workers always encounter systematic asymmetries in workload across time, space, and accounts, which are caused by opaque AI-driven allocation rules. The dispatching mechanism often causes location-based platform workers either to undertake unbearable juxtaposed workload or to experience irregular schedules with unwanted idle time, making the labour process full of frictions. This paper empirically investigates the labour process of food-delivery riders on Meituan, China’s largest food-delivery platform, in Shenzhen. Drawing on an 11-month hybrid ethnography, including four months of participant observation as a rider on my own and 65 in-depth interviews, I explore situated collective practices rather than isolated individual ones that compensate where AI-enhanced technologies remain dysfunctional. My paper shows how workers make sense of such an unpredictable, unstable and uneven dispatching mechanism and keep it running in everyday logistics: by collaboratively re-circulating orders within groups accompanied by timely online communication and manual reallocation of human intermediaries on the backend. It further shows that constant yet hidden interpersonal negotiations on the ground, sustained by workers’ emotional and relational labour, play an indispensable role in making the high-speed delivery possible. The paper reveals the limits of AI-enabled automation and suggests that techno-economic systems operate by embedding in and entangling with sociocultural logics rather than being vacuously omnipotent.
Paper short abstract
This paper claims that users attend to problematic output in chat with LLMs with methods that they also employ in conversations with other humans, but these methods get subtly transformed in ways that helps us to understand what we are faced with in interactions with AI.
Paper long abstract
We know since ELIZA that human users will attempt to make sense of the output of a chatbot as a response to their own contributions to the chat irrespective of the technological foundation of such a system (Weizenbaum 1967; Eisenmann et al. 2023). While ELIZA was most successful when users imagined a psychiatric context and rely on a limited repertoire of conversational actions, large language models (LLMs) are much more flexible in the roles they can be instructed to play (Shanahan et al. 2023) and the conversational actions that they perform (Stokoe et al. 2025), but we can still observe how users find sense in generated outputs and disregard inappropriate output (Pütz 2025). At the same time, there are occasions where users attend to problematic model outputs. To discover such interactions, I use a large dataset of 2 million chat interactions that is available through the WildVis search tool (Deng et al. 2024) and collect instances where users question or disagree with model outputs and consider the sequential context of these occasions. These instances are compared to and contrasted with findings from conversation analysis concerning disagreements in interactions among humans. I will suggest that users attend to relevant problematic output in chat with LLMs with the methods that they employ in conversations with other humans, but these methods get subtly transformed in ways that help us to understand what sort of human-machine interaction we are faced with.