Send message to Author

Paper short abstract

The reliance on Large Language Model(LLM) outputs as evidence by 'non-human informants' challenges pre-digital adjudicatory frameworks. To bridge this, I propose the Explainable Audit Trail(XAT), a post-AI reconceptualisation of the audit trail, built on empirical and interdisciplinary research.

Paper long abstract

Generative AI systems, such as Large Language Models (LLMs), are increasingly used in the justice system in England and Wales to process forensic audio and textual data, performing tasks such as transcription, translation, summarisation, and interpretation. Their outputs may serve as ‘expert evidence’ that judges and juries must evaluate, positioning LLMs as ‘non-human informants’. In this sense, adjudication resembles qualitative inquiry, relying on interpretation, triangulation, and assessments of adequacy, albeit within the safeguards of trial fairness.

LLMs introduce familiar challenges, like inaccuracy and opacity, yet traditional mechanisms for testing reliability, such as cross examination and summative reports are poorly suited to them.This underscores the need to reconceptualise how AI outputs are assessed for reliability in criminal adjudication.

This research draws on audit trails, a long-standing tool for documenting how information is created and interpreted across disciplines, including digital forensics and qualitative research. With roots dating back to the fifteenth century, audit trails provide a pre-digital foundation for evaluating ‘non-human informants’.

I propose the Explainable Audit Trail (XAT): a reconceptualisation of the traditional audit trail designed to enhance the reliability assessment of AI systems. Grounded in empirical analysis of digital forensic practice and interdisciplinary scholarship across law, human–data interaction, explainable AI, and scientific communication, XAT provides process transparency across the evidential lifecycle. It documents how LLM outputs are generated and interpreted, enabling courts to assess reliability in a structured way. Through XAT, I demonstrate how pre-digital methodologies can support post-digital evaluation of LLM outputs.

Send message to Author

Paper short abstract

How can qualitative inquiry remain reflexive when working with algorithms, archives, and AI outputs? The Extended Digital Case Method treats non-human informants as generative analytics partners while centering researcher judgment to preserve contextual integrity when interpreting digital outputs.

Paper long abstract

How can qualitative inquiry remain reflexive when informants include non-human actors such as algorithms, archives, and AI outputs? This paper develops the Extended Digital Case Method (EDCM), a conceptual framework that positions digital actors as methodologically consequential “non-human informants” rather than substitutes for immersion. Building on Michael Burawoy’s extended case method, which emphasizes theory reconstruction through empirical anomalies, the EDCM adapts reflexive ethnography to the epistemic and methodological challenges of digitally-mediated fields. Platform governance, temporal persistence, and high-volume interactions complicate access, participation, and interpretation, while computational tools risk abstracting social relations from context.

The EDCM addresses these challenges by integrating digital tools–APIs, archives, and small language models–as extensions of reflexive practice. The researcher remains the accountable agent, using these tools to surface anomalies, extend temporal observation, and interrogate deviations while preserving interpretive fidelity to situated social practices. By combining sustained participant observation with selective digital augmentation, the EDCM operationalizes pre-digital principles in post-digital settings: adequacy for scope, contextual sensitivity for depth, and iterative engagement through repeated analysis for validity. Computational outputs function as generative informants whose contributions are interpreted and reintegrated through human judgment.

Illustrated through a digital ethnography of r/Antiwork, a 2.9M-member Reddit community focused on labor critique, the EDCM reveals minority perspectives, platform-mediated governance, and interactional divergences that challenge traditional labor and social movement theories. More broadly, it demonstrates how qualitative inquiry can maintain analytical rigor, accountability, and theory-generative potential as human and computational actors jointly shape social knowledge, situating post-digital methods within a lineage of reflexive ethnography.

Send message to Authors

Paper short abstract

This paper opens the black box of text embedding benchmarks, revealing that “semantic similarity” is not a settled matter of fact but an assemblage stabilized by alignment logics foreign to SSH. We propose descriptive benchmarks as an algorithmic resistance intervention.

Paper long abstract

Embedding models, employed in retrieval-augmented generation techniques that extend the capabilities of large language models, are said to encode (i.e. tasked to perform) “semantic similarity.” But in what sense, exactly, and can it be repurposed for research in the social science and humanities (SSH)? Benchmarks such as MMTEB (Enevoldsen et al., 2025) de facto set this standard.

In this paper we review the benchmark literature, observe that most benchmarks approach semantic similarity as a matter of human alignment, and contest that such a frame is necessarily the most relevant to assess SSH applications of embedding models. To defend the methodological autonomy of SSH and as an algorithmic resistance intervention, we propose a different approach to their benchmarking, departing from the alignment problem, and accepting to live with the trouble of humans not being aligned with each other in the first place. Rather than setting a single universal standard, we defend descriptive benchmarks that document analytically useful yet potentially incompatible features that can never be assembled into a single scale from worst to best model.

First, we unpack how MMTEB compiles benchmarks that in turn compile smaller benchmarks, and discuss what this aggregative logic performs on the nature of semantic similarity. Second, we articulate our approach to “descriptive” benchmarks in the case of an application to observatorial controversy mapping. Third and finally, we showcase an implemented example that documents whether a model considers two opposite opinionated statements less similar than with a neutral one (and find that many do not).

Send message to Author

Paper short abstract

This paper offers qualitative researchers an adjudicative vocabulary for LLM-assisted inquiry, drawing on a tripartite framework of affordances, experience, and conceptualization to interpret, situate, and adjudicate the epistemic status that AI-generated material holds within qualitative inquiry.

Paper long abstract

Qualitative researchers are rapidly adopting Large Language Models across different stages of inquiry. However, the conditions shaping their methodological use are often left unstated. A growing body of work shows LLMs can assist qualitative work while intensifying questions of judgment, opacity, and responsibility. This paper focuses on a narrower problem: how qualitative researchers interpret, situate, and account for AI-generated material.

Instead of treating AI as a neutral tool or a non-human informant, the paper asks how AI-generated text becomes part of inquiry, since it may function as heuristic prompt, condensed summary, coding aid, draft language, or evidence. In prior research, the author developed a tripartite framework for analyzing relations to technology through affordances, experience, and conceptualization. That research showed that digitally mediated objects are rarely accessible directly but are approached through practices shaped by the technologies that enable them and their social contexts.

This framework is applied to specify what LLMs afford researchers, how working with such material is experienced, and how its epistemic status is conceptualized in the field. The contribution is not a new method but an adjudicative vocabulary for describing and structuring how researchers decide what status AI-generated material holds within inquiry. This matters because each new technological capacity reshapes individual experience and collective sense-making, and chat-based LLMs intensify this process by presenting themselves as open-ended assistants while leaving norms of methodological use unsettled. The argument connects grounded theory's concern with analytic fit and ethnomethodology's focus on contextual interpretation with the interpretive demands of AI-supported qualitative research.