Send message to Author

Long abstract:

Since the 1960s, numerous academic, industrial and military research projects have gone beyond the conceptualisation of computer based systems as tools for extending human capabilities or controlling mechanical operations and have programmatically conceived and designed them as cooperative partners that actively participate in work processes. Computers were envisioned to support human work by taking over certain clerical tasks or even being directly involved in decision-making processes together with human operators.

Although ideas conceived in the 1960s such as “man-computer symbiosis“ or “man-computer partnership“ (Licklider 1960; 1964) seemed futuristic at the time they however still have an impact on the development of agentic media today and resonate with contemporary research approaches like human-autonomy teaming (cf. Lyons et al. 2001). In light of recent developments in the fields of generative machine learning, artificial intelligence and interface design, as well as the proliferation of sensor technologies, many futuristic visions of the 1960s and 70s appear to be technically feasible today. In context of these developments the paper aims to re-examine the historical concepts of human-machine cooperation and evaluate them based on their socio-technical assumptions, role expectations, and autonomy concepts.

Drawing on relevant literature and debates in cybernetics, human-computer interaction and human-autonomy teaming, the paper traces the media historical development of agentic media as co-operative other(s) since the 1960s. In doing so the paper aims to unpack the developments underpinning today’s proliferation of synthetic agents and discusses how seemingly historical concepts can contribute to the understanding of contemporary technological developments.

Send message to Authors

Long abstract:

We compare the communicative aspects and forms of agency of LLM-based systems for text generation with those of traditional search engines. Systems such as Google search practically achieve the automation of an extensive and dynamic catalog. Such a system allows users to enter into communication with authors (who may be also unknown, anonymous or multiple) of texts that can be accessed by following links provided in response to their queries. The communication partners are still human beings. LLM-based systems such as GPT3.5 instead, use the texts produced by human beings in the training data and in the interaction to autonomously generate content that may have never been written before and read by a human. In this unprecedented form of artificial communication, the communicative partner is directly the machine and the agency of such systems can be understood as an interactional achievement .

The new, A.I.-powered search engine Perplexity combines the features of both search engine and generative A.I. (more generally described as “retrieval augmented generation”), operating as a partner that enables communication both through others and with others. Through empirical experiments, we compare the performance of Perplexity with that of GPT3.5, focusing on the interaction with the user in response to factual questions, the use and indication of sources, and the handling of hallucinations or lack of knowledge.

Our hypothesis is that the communicative agency of the digital system underlies Perplexity's performance, enabling it to achieve results that can exceed those of systems much larger in size and computational capacity.

Send message to Author

Long abstract:

This contribution elaborates on recent attempts to advance the project of artificial (general) intelligence based on my reading of emerging research in the field from the perspectives of media studies and media philosophy. I will focus on the instrumentalisation of worlding (setting up of the world) in machine learning, that is, blurring of the boundaries between the learning subject and its environment for the purpose of modulating synthetic agency. I will discuss several articles that outline novel AI architectures or virtual environments for ML research and converge on the epistemic rendering of what the “world” is.

My hypothesis is that the current project of A(G)I can be understood relationally through an analysis of what I refer to as “model worlds” and “world models”. Model worlds are game-like simulation toolkits assembled for the purpose of ML research, — expanding on Bruder’s (2021) writing on the use of microprocessors as model organisms in neuroscience, — while world model is a concept in ML that refers to AI agent’s capacity to represent the world’s dynamics (Ha and Schmidhuber 2018). The technical sources figure the “world” either in the sense of algorithmic knowledge domains or in relation to AI agent’s directive to grasp the world. Whether external or internal to agents, such simulations of the world allow for predictive control and, therefore, the governance of potentialities, including the drive to learn. Through the dialectics of *world* in this AI/ML research corpus I aim to demonstrate how synthetic agency is engineered, negotiated, and reconfigured.

Send message to Author

Long abstract:

Following the release of ChatGPT in November 2022, generative AI quickly moved to the center of technical imaginaries worldwide. Developments and news in this field are rapidly evolving in a competition among global technology companies to create increasingly powerful language models, integrate them into specific practical contexts, and explore ways to commodify them. Interactions with language models are conceived as dialogs with AI agents, which can be fine-tuned in various ways and operate semi-autonomously through different interfaces in digital environments. AI researchers working on large language models argue that “agenticness” (Shavit et al. 2023) is what’s specific and new about applications such as ChatGPT, Gemini, Copilot etc. With the introduction of GPTs and a dedicated GPT Store by OpenAI, we are currently experiencing an explosive proliferation of AI agents, raising the question of which differences among AI agents make a difference and how these differences manifest in media practices. Against this backdrop, the paper attempts to map the dynamically evolving field of AI agents. Opposing the rhetoric of the radically new, it first recalls the recurring idea of software agents in the history of AI. Secondly, drawing on approaches from digital methods it explores experimental approaches of engaging with the multiplicity of AI agents and their agentic capacities in situ.

Send message to Author

Short abstract:

Artificial intelligence and machine learning techniques mediate a vast portion of contemporary social interactions. Based on the analysis of patents from Google, this work examines the social action of machine learning and the consequent tensions in the modern notions of humanity and agency.

Long abstract:

The recent success of neural networks and deep learning (LeCun, Bengio, and Hinton, 2015) revamped not only the cultural and political discourse around AI but also the effective use of machine learning (ML) techniques in diverse aspects of everyday life. Big tech companies make intense use of ML for automation, anticipation, and preemption in online platforms, smartphone apps, and other digital technologies. Particularly, Google can be seen as the first and most relevant actor in an accumulation logic based on big data and AI (Zuboff, 2015). In parallel, advances in humanities and especially in science and technology studies begin to question the modern paradigm of humanity and its exclusivity of agency (Haraway, 1991; Latour, 1991; Viveiros de Castro, 2009).

In this context, this work draws from an analysis of patents from Google to describe the social action of ML and its implications for current notions of humanity and agency. Based on the notion of technical mediation (Latour, 1994) we propose to conceptualize ML in terms of algorithmic mediation. This mediation occurs through processes of capture (of information into digitalized data) and modulation (of user activity through informational inference/incidence and environment modification). In this way, humanity circulates between human and non-human actors, and action is produced by an heterogeneous assemblage. Our conclusions corroborate previous analyses that relate ML agency and the reproduction of the past, creation of stereotypes and discrimination (Benjamin, 2019; Crawford and Paglen, 2019; Gebru, 2020; Pasquinelli and Joler, 2021) resembling an automated and preemptive sociability.

Send message to Authors

Long abstract:

Machine learning is itself an instrument - a technical apparatus which can serve to sense and present, and make available to interaction that which could not be engaged before. But machine learning applications themselves operate downstream of instruments, reliant on the long pathways of sensors that generate data, and which in turn configure ML and feed its algorithmic analytics. This paper thus explores ML as concatenations of instrumentation, or recurrent cascades of sensing, presenting, and then sensing and presenting again.

Instruments (and not only algorithms and data), deserve our attention. Instruments differentiate and materialize the world according to their design, though not so much as to be ‘determined’. By definition an instrument must be capable of surprise, of revealing something other than what was expected or even hoped, though not so much so as to produce incoherence, a finding that cannot be placed and understood.

We explore the matter through the case of energy generating windmills. These windmills consist of an aggregation of components (eg. gearboxes, transmission, turbines), each of which are themselves replete with data generating sensors, and in turn these data are fed into machine learning models (eg. for predicting maintenance needs or energy output). In sum, sensors layered on sensors, which at each stage are themselves agentic, that is, consequential in how they intermediate phenomena, presentation and interaction.

Send message to Author

Long abstract:

Recent advancements in deep and machine learning have shifted technology's conceptualization and utilization, surpassing its traditional mediatory role. This shift prompts questions of how agency is distributed between humans and synthetic agents, of how agential relations between humans and technologically developed agents are produced, and how agency is transformed. Based on an ethnographic case study in robotic laboratories, the paper investigates how agency is distributed and differentiated within the manufacturing processes of robots. In robotics, agency is assumed to be implemented, with humans implementing agency in technology, thus leading to a differentiation between human-related implementing and technological-related implemented agency.

While previous STS research has focused on concepts of ascription and inscription of agency, the notion of implementation has received less attention. The contribution heuristically explores the interplay between implementing and implemented agency within the realms of assimilation and deviation. For instance, it analyses scenarios where robots defy the expectations of roboticists, revealing a form of recalcitrance. By focusing on the implementation of agency, the structural and material elements take on a central role in shaping agents as either technological counterparts or human-analog interactants and participants. The paper aims to contribute to the question of how far agency is transformed in processes of its implementation in situ, but also in how it transforms such processes by releasing a differentiating potential between roboticists and robots. By shedding light on these transformation processes regarding a transformed-transformative agency in robot laboratories, it contributes a deeper understanding of human-machine interaction and its modes of participation.

Send message to Authors

Long abstract:

The complex organization plays a huge role in transformation theories that explain the transition from a premodern to the modern age, it is in a sense the tool for the coordination of human groups that allowed industrialization and modernity to take hold. Max Weber pointed to hierarchy and expert knowledge as two mechanisms that explain the effectiveness of complex organization. Laboratories, both big and small, are organizational units, but neither hierarchy nor human individual intelligence best characterize Labs, as STS has shown—hierarchy is often played down, non-existent or not possible, intelligence appears distributed, and object relations—rather than personal relations—dominate in human labs. This paper takes this assessment as a starting point for investigating a further transformation, that from human to human-free laboratories in advanced autonomization efforts in which the problem of human coordination empties out and is replaced by the goal of creating the self-driving artificially intelligent agentic setting for science.

This paper explores current autonomization efforts with a focus on disciplines in which such efforts are most advanced, such as chemistry and biology labs.

Send message to Author

Long abstract:

As in other fields, machine learning (ML) tools are promising a technological solution to educational problems. Tasks that are considered central to teaching are increasingly automated by ML tools: Intelligent tutoring systems provide feedback and select appropriate exercises for individual students (task selection), automated grading evaluates essays and open-text exercises (assessment), and learning analytics promise to predict student performance based on their past actions tracked on learning platforms (diagnostics). This not only raises questions about the professional identity of teachers, but also disrupts the established distribution of agency between humans and machines in education. The proposed paper addresses this (contested) shift in the distribution of agency using empirical examples from an ethnographic project on the use of ML tools by teachers in Swiss secondary schools. Three dimensions of agency are identified that are particularly affected by ML tools: (1) expertise: in the face of (seemingly) objective tools, teachers are expected to justify their pedagogical decisions when they deviate from automated systems; (2) autonomy: consequently, teaching often involves defending the autonomy of the profession against the claims of companies promising better results; (3) accountability: Finally, the advent of ML tools in education also obscures questions of accountability by distributing it across a range of actors (teachers, administrators, developers, systems...) - who should one turn to when grades are perceived as unfair or learning outcomes as lacking? Conceptually, the paper draws on STS research on ML, mediated accountability, distributed agency, and the sociology of professions.

Send message to Author

Long abstract:

In different forms of art, a growing number of artists are taking advantage of generative AI technologies to transform their creative practices, delegating different degrees of agentive control and artistic decision-making to those technologies in the hopes of finding inspiration in their output and thereby expanding their own creative horizons. This paper focuses ethnographically on David, a computer scientist who is also a semiprofessional jazz trumpet player, and on his development and use of a jazz-improvising digital interactive system in an Institute of Technology in the US. It explores the programming decisions made by David in his attempt to transform the system’s creative agency in specific ways, and how those decisions resulted in equally consequential transformations in David’s own creative agency as a result of his joint improvisations with this system. In doing so, the paper points to the co-constitution of human creative agency and machine creative agency in the age of generative AI.