Send message to Author

Short abstract:

To enable meaningful contestation of public algorithmic systems, lessons can be learned from democracy, the Rule of Law, and system safety. These disciplines show the need for institutional and organizational cultures that reinforce feedback channels in design processes of public algorithmic systems

Long abstract:

Public organizations are increasingly confronted with and held responsible for harms emerging from their public algorithmic systems. Meanwhile, the design processes of these systems are in an institutional void, shifting design activities to subpolitical realms and impeding affected citizens to contest these systems. Currently, design processes are advanced by developing and implementing policy instruments that should support the ethical, legal, and technical scrutinization of the algorithmic systems being designed, for example, impact assessments and algorithm registers. Although these instruments do provide information flows needed for contestation, they represent an ad hoc and untargeted approach that does not guarantee meaningful contestation. Instead, contestation needs to be part of feedback channels that are institutionalized in the design process of public algorithmic systems. In this contribution, I explore and combine the insights on institutionalizing feedback channels from disciplines representing the nature of public algorithmic systems: democracy, the Rule of Law, and system safety. Democracy and the Rule of Law, structuring public administration contexts, consider citizens to be the central actor in contestation advanced in a system of checks and balances. System safety considers hazards in software-based systems as emergent properties that can only be controlled through learning in feedback channels. In that case, contestation is a starter for system quality improvements. An important similarity between these perspectives is their emphasis on specific institutional and organizational cultures that acknowledge, value, and support contestation. In this contribution, we will describe the conditions for such a culture in public algorithmic design processes.