Send message to Authors

Paper short abstract

Although the systemic risks of AI are already subject to regulation, there is a lack of understanding their systemic nature, characterised by emergent effects at a societal level, effectively hindering adequate assessments, mitigation measures and governance.

Paper long abstract

The integration of general-purpose AI models into downstream AI systems has given rise to new forms of risks that are more systemic in nature than conventional AI risks. Although these risks gain traction in research and regulation, there are widely varying understandings of them. In the EU AI Act for example, systemic risks can be caused by individual AI providers or users. However, we are already observing systemic risk phenomena arising from complex interactions between different actors or AI systems, threatening societal goals. Yet the ‘systemic nature’ of these phenomena is not adequately addressed by current risk assessments, mitigation measures or governance frameworks.

We propose to investigate systemic risks of AI as arising from complex interactions between system elements (e.g. data and inference transfers), resulting in emergent effects at a societal level, such as the gradual erosion of human rights (e.g. systemic discrimination), democracy, and environmental protection goals. This conceptualisation views systemic risk phenomena through the lens of complex externalities and collective action problems. Of particular interest are feedback dynamics, processes that lead to market concentration like network effects, algorithmic monocultures, and integration processes of AI supply chains or ‘AI ecosystems’ and across societal sectors, which can result in structural dominance, dependencies, and cascading risks. We also outline regulatory deficits and normative ambiguities that hinder effective mitigation efforts. Finally, we discuss how these phenomena of systemic risks interact and accumulate to create future challenges.