(A27)
The power of correlation and the promises of auto-management. On the epistemological and societal dimension of data-based algorithms
Location Welcome Centre Lecture Theatre 2
Date and Start Time 27 Jul, 2018 at 09:00
Sessions 3

Convenors

  • Jutta Weber (University Paderborn) email
  • Gabriele Gramelsberger (RWTH Aachen University) email

Mail All Convenors

Short abstract

We will analyze the epistemological and ontological groundings of data-based learning algorithms as well as reflect how the logic of data-driven algorithms and the increasing automation of decision-making corresponds to our societal condition.

Long abstract

Data-based learning algorithms do not only open up new dimensions of knowledge production but shape increasingly more highly relevant societal decision-making processes. Based on correlation-based search heuristics, these algorithms are used to improve translation programs, search engines, predictive policing strategies or so-called ‚adaptive‚ intelligent' protheses. They are highly relevant in realms ranging from financing, administration and technoscience, law enforcement and counter-insurgency. These algorithms do not only embody sociotechnical practices of human and non-human actors but also transport invisible and unquestioned values, norms and preferences.

In our panel, we want to analyze the epistemological and ontological groundings of data-based learning algorithms - how, for example, they are dependent on the provided / selected data material, on implemented classifications, categorizations and problem-solving strategies. At the same time, we want to reflect how the epistemologic logic of data-driven algorithms and the increasing automation of decision-making corresponds to our societal condition: We want to ask whether correlation does not only feed into the datafication of our world but also serves a contemporary sociocultural and biopolitical logic of risk management or whether the increasing automation of decision-making reflects post-democratic developments in the Global North.

This panel is closed to new paper proposals.

Papers

The rise of big data (technologies) as Eurocentric and androcentric endeavor

Author: Bianca Prietl (Technical University Darmstadt) email
Mail All Authors

Short abstract

Reconstructing the epistemological and ontological assumptions of big data analysis as put forward by its key proponents, this paper argues that the rise of big data and data-based learning algorithms can be understood as a Eurocentric and androcentric project.

Long abstract

This paper argues that the rise of big data and data-based learning algorithms can be understood as a Eurocentric and androcentric project. The argument is unfolded in three steps: First, it is shown that the epistemological and ontological assumptions underlying big data analysis draw on an understanding of objectivity that has been established in 18th and 19th-century science. Second, this modern understanding of objectivity as neutral observation 'from nowhere' is shown to be inextricably linked with the constitution of the modern subject of reason as a 'white', masculine subject. Based on these first two analytical steps it is then argued that there are important historic continuities in the development of big data and data-based learning algorithms that are easily overlooked when considering big data as radically new technological development. These continuities point at big data analysis and data-based learning algorithms as the current manifestation of a logic of reasoning with specifically modern roots.

The empirical basis for this argumentation are statements on and assumptions about big data analysis that have been made publically available by key actors and proponents in the field. Theoretically the argumentation is foremost inspired by feminist technoscience, but also critical data and algorithm studies and social studies of numbers and accounting.

Learning in the wild: on the problem of adaptivity in machine learning

Authors: Nikolaus Pöchhacker (Technical University of Munich) email
Marcus Burkhardt (University of Siegen) email
Mail All Authors

Short abstract

The promise of machine learning applications is to be able to adapt to unforeseen futures without being explicitly programmed. This proclaimed adaptivity is, however, not an automatism. We therefore ask how adaptivity is accomplished in machine learning on different levels and to varying extents.

Long abstract

In June 2017 Sundar Pichai, CEO of Google, proposed a paradigm shift in the history of computing: Innovation should neither be driven by approaching problems as first and foremost digital nor mobile, but instead by taking an AI first approach that is fueled by recent advances in the field of machine learning. This statement reflects a central promise of machine learning applications, namely the ability to adapt to unforeseen futures without being explicitly programmed: visual recognition of objects or persons without ever having seen or trained on this specific object or this specific person before, self-driving cars being able to deal with new situations safely or chatbots conducting conversations with humans in an engaging manner.

Conversely, the more such technologies are built into the fabric of everyday life the more concerns are raised about their potential risks, e.g. biases and inequalities inherent in training data sets. As a result, ML models often produce (social) structures instead of adapting to them. This tension between promises of ML and perceived risks points toward a hitherto largely unstudied aspect of data-driven applications: the production of adaptivity in real-world ML applications. Drawing on examples like Microsoft's chatbot Tay.ai, recommender systems and fraud detection applications the paper aims to unpack the notions of adaptivity that ML rests upon. By focusing on how adaptivity is accomplished on different levels and to varying extents our goal is to explore the ontological politics that ML systems enact in the wild of their real-world deployment.

Exploring the bias in de-biasing

Author: Doris Allhutter (Austrian Academy of Sciences) email
Mail All Authors

Short abstract

My paper analyzes practices of de-biasing in machine learning and natural language processing. It investigates the concept of bias that different de-biasing methods are based on and shows how differing ideas of gender bias and racial bias suggest solutions that vary widely in complexity.

Long abstract

In the past two years, researchers in machine learning and natural language processing have put much effort into finding ways of removing gender bias and racial bias produced by classification learning algorithms and word embeddings (e.g. Berendt & Preibusch 2017; Bolukbasi et al. 2016; Caliskan et al. 2017). Computer scientists experiment with different solutions and contexts and are gaining deeper insight into how profoundly data - text, language and societal discourse - are gendered and entangled with racist stereotypes. However, when exploring methods for de-biasing, computer scientists are also actively taking part in the co-production of meanings surrounding social categories such as gender and race.

From a technological point of view, de-biasing is not a trivial task and the research that is done to avoid amplifying "human-like biases" seems to be a driver to improve machine learning as a whole. However, the problem is by and large diagnosed to exist in society and technical solutions are to adjust human shortcomings. My paper analyzes practices of de-biasing in machine learning and natural language processing. It investigates the concept of bias that different de-biasing methods are based on and shows how differing ideas of gender bias and racial bias suggest solutions that vary widely in complexity.

Berendt/Preibusch (2017). In Big Data 5(2), 135-152.

Bolukbasi at al. (2016). In Advances in Neural Information Processing Systems, 4349-4357.

Caliskan et al. (2017). In Science 356(6334), 183-186.

Auto-management as governance? Predictive analytics in counter-insurgency and marketing

Author: Jutta Weber (University Paderborn) email
Mail All Authors

Short abstract

In my paper I will analyze whether and how the automated management and decision making can be interpreted as a new way of governance. Therefore, I will compare applications of predictive analytics in counter-insurgency and marketing.

Long abstract

Automated management is build on the capture, structuring and sorting of data through algorithms but also the reshaping of human behaviour to make it amenable to the automated, automatic or autonomous systems of datafication.

In my paper I will analyze whether and how the automated management and decision making can be interpreted as a new way of sociocultural and biopolitical governance. Therefore, I will compare applications of predictive analytics in such diverse fields as counter-insurgency and marketing.

Whose knowledge, whose power? Investigating principles of machine learning from a feminist epistemological perspective

Authors: Goda Klumbyte (University of Kassel) email
Claude Draude (University of Kassel) email
Mail All Authors

Short abstract

This paper looks at the principles of knowledge production and legitimization through data-based algorithms in machine learning. It asks what kind of conceptual models of learning and knowledge are proposed, and how these models can be re-evaluated from the perspective of feminist epistemologies.

Long abstract

This paper will look at the principles of how knowledge is produced and legitimized through data-based algorithms in machine learning. We will interrogate what kind of conceptual models of 'learning' and 'knowledge' are proposed through algorithmic processes of machine learning, and how these models can be evaluated from the perspective of feminist epistemologies. Specifically, we will investigate how data-driven machine learning and resulting knowledge production is based on principles of abstraction, categorisation and correlation.

As technological as well as discursive principles these concepts served as foundational in pursuit of modern science and production of objective, truthful scientific knowledge. However, feminist epistemologies and critiques of science have pointed out that such knowledge production is closely related to unequal power dynamics that exist in a given society, and argued for a more embedded, embodied and situated perspectives on ways of knowing and legitimization of truth claims. Taking these aforementioned critiques into account, we will ask how data-driven algorithmic knowledge production in machine learning can be re-read in ways that account for the power dynamics that are re/produced through the models of 'knowledge' and 'learning' that are at the core of such knowledge production.

Trust and machine learning

Authors: Hendrik Heuer (University of Bremen  ) email
Andreas Breiter (University of Bremen) email
Mail All Authors

Short abstract

We discuss the challenges that arise when people interact with machine learning systems. Considering the complexity and indeterminacy of such systems, we argue that it is impossible for people to consciously reflect on machine learning. We further argue that trust helps to overcome these challenges.

Long abstract

If a pen runs dry or an internet connection dies, people experience a breakdown in the use of their tools. Such breakdowns cause a shift of focus that reminds people of the discrepancy between their actions or expectations and the world (Winograd and Flores, 1986). In Heideggerian terms, the pen or the internet connection changes from "ready-to-hand" to "present-at-hand". When something becomes "present-at-hand", people consciously reflect on it. We want to understand what it means for a machine learning system to be "present-at-hand". Machine learning systems pose a challenging problem: Not only are interactions with them necessarily mediated, machine learning systems like neural networks are inherently complex and it is impossible to evaluate them comprehensively. A French-English translation system translates infinitely different French sentences into infinitely different English sentences. Assessing the quality of such a system and consciously reflecting on it is impossible for a user since it requires understanding a machine learning system's inner logic and testing every input-output combination. Despite this indeterminacy, people successfully interact with machine learning systems all the time. We believe that trust is what makes this possible. Trust enables people to face the complexity of organizations, other people, or abstract things like money and political power. We argue that trust also enables people to interact with machine learning systems since trust allows people to face uncertainty, manage complexity, and take risks.

PRECOBS: theory- and correlation-based construction of crime futures

Author: Simon Egbert (Universität Hamburg) email
Mail All Authors

Short abstract

Predictive policing is not as revolutionary as commonly depicted. Rather, corresponding software uses quite conventional sources of information. I will illustrate this by drawing on empirical data of the development and utilization of the crime prediction software 'PRECOBS'.

Long abstract

For several years now, digital technologies mainly consisting of methods of data mining and algorithmic decision making, commonly referred to as 'predictive policing', are employed by police departments throughout the world. When scrutinizing practices of predictive policing in detail, it becomes apparent that they are not as revolutionary as commonly depicted. Rather, such software tools mostly draw on criminological theories (e.g., rational-choice approaches) and a probabilistic approach to police data, which represent quite conventional sources of information. Hence, predictive policing actually does not foresee the future but (solely) helps police to identify potential future high-risk areas, by using historic (crime) data, interpreting these by using ordinary theories of crime and by correlating these insights again with present crime situation information, eventually transferring them algorithmically into actionable knowledge.

In my presentation, I want to describe the human-machine-interaction of predictive policing in more detail, especially with reference to the embodied algorithmic values and norms as well as to the utilized data material. I will reconstruct the sociotechnical construction of crime futures by drawing on empirical data of the development and utilization of the crime prediction software 'PRECOBS', the leading crime prediction product in German-speaking countries.

Another point I aim to stress: Especially because of their enablement and/or simplification of crime data analysis in general, predictive policing is unlikely to be a short-dated phenomenon but a strategy which is presumably to be an important part of police work in the future, ultimately giving rise to the 'datafication' of police work.

Continuities and discontinuities in the governance of motorized and autonomous traffic

Author: Cordula Kropp (Uniuversität Stuttgart) email
Mail All Authors

Short abstract

The paper will discuss continuities and discontinuities in the development of approaches to the technical governance of motorized and "smart" cars.

Long abstract

The call states that data-based learning increasingly shapes social decision-making processes and that the algorithms used explicitly embody and implicitly transport sociotechnical norms and rule-sets. At the same time, existing sociotechnical practices and rule-sets are being changed through interaction with machine learning. At present, this is particularly observable in the field of self-driving cars and the intertwining of social learning and machine learning: "Society is learning about the technology while the technology learns about society" (Stilgoe 2017: 1). A crucial question is, how the ways of dealing with responsibility will be reconfigured.

Against this background, a comparison shows how human responsibility, technology governance and legitimate decision-making have become established in the industrial phase of the incipient mechanization of automobile traffic, and to what extent today's approaches towards the governance of smart mobility differ from it. Based on an analysis of socio-technical imaginations and their current impact on decision-making in Germany my contribution will reconstruct astonishing continuities with, however, some significant differences.

Stilgoe, J. (2017): Machine learning, social learning and the governance of self-driving cars. In: Social Studies of Science. Online first.

Shaping haptics, meeting humans: data-clustering algorithms in human-robot-collaborations

Authors: Dawid Kasprowicz (RWTH Aachen University) email
Gabriele Gramelsberger (RWTH Aachen University) email
Mail All Authors

Short abstract

Data-Clustering algorithms are essential in human-robot-collaborations. However, they have been treated marginally so far when it comes to the social dimension of robots. The paper shows how technical and social dimensions intertwine in the modeling of haptics in human-robot collaborations.

Long abstract

In the last couple of years, the notion of the social in "social robots" has become highly disputed. One the one hand, most articles from robotics emphasize aspects of humanoid design and companion-like behavior as social factors. On the other hand, scholars from the Social Studies of Science focused upon the technological situatedness of these non-human-objects in everyday practices - whether as a maintenance of bodily interactions (M. Alač) or through the constitution of a dialogical space in a network of agents (R. Jones).

In all these approaches, however, the impact of data-based algorithms for the creation of "the social" has been treated marginally. A good example for this is the most important sense for human-robot-collaboration: haptics. Focusing on case studies coming from the coding of haptics for human-robot collaborations, the paper argues that data-clustering algorithms are highly dependent on both the engineering ideal of an "unsupervised learning algorithm" and the formalized expectation of social responsiveness. In order to structure the data of the robots' sensors and to process a socially anticipated movement, the algorithm constructs a virtual human, a human to come, as a formalization of the social responsive body. Therefore, this construction of algorithms should be considered as both - a factor of computationally driven models of bodily interaction and a social practice of defining the virtual human to come. This means to design the human as the environment of the robot.

This panel is closed to new paper proposals.