Send message to Author

Short abstract

This paper uses Star & Gerson's sociology of anomaly management to explain how "rationalist" forum LessWrong rally to investigate "glitch tokens"—disruptive and unexplainable AI outputs that defied explanation, through a combination of engineering, "media archaeology" and speculative myth-making.

Long abstract

When specific string tokens (such as "SolidGoldMagikarp" or " petertodd") were shown to trigger bizarre, evasive, or ominous behaviors in GPT-3, users of the "rationalist" web forum LessWrong faced an undocumented practical problem: classifying and investigating outputs that defied the model's advertised capabilities. This paper examines the community's situated reasoning practices, drawing on Star and Gerson's (1987) framework of anomalies as negotiated, highly situated interruptions to routine work—classified through debate as mistakes, artifacts, discoveries, or improprieties depending on organizational context and power relations.

Through a situational analysis of the "Glitch Tokens" thread (2023–2025), the analysis surfaces the community's practical epistemic labor, the ad-hoc methods and various forms of proficiencies deployed (Reddit "archaeology", repeated queries, etc.), and the competing interpretations stemming the community's distinct "social worlds"— ranging from "mechanistic interpretability" practitioners suspecting an embedding fluke to self-described "cyborgists", open to dialogue with the transcendent entities they perceived within the weights : the benevolent "Leilan" or the disruptive "petertodd". Crucially, when OpenAI silently patched these behaviors, members described this as "lobotomy," observing that the model now "writes scared." This anthropomorphic framing accomplishes important boundary work: by conflating the model's silenced interiority with their own stifled epistemic access, LessWrong members reclaim interpretive authority over a product whose access is contingent on the company's goodwill.

By attending to these heterogeneous methods of anomaly classification—whether grounded in credentialed expertise or speculative encounter—this paper underlines the crucial role of folk theories and exploratory practices upon the contested arenas of AI adoption.