Send message to Authors

Paper short abstract:

AI heralds a future requiring risk governance, and technological safety hinges on choices shaped by economic incentives. While the impact of AI on the economy has been highlighted, little is said about the impact of the economy on AI development and its economisation.

Paper long abstract:

The capabilities and affordances of ‘frontier’ Artificial Intelligences (AIs) are growing at a rate that would scarcely have been credible five years ago. These advances undoubtedly herald a transformative future, but the exact nature of that future remains an open question wherein great promises are intermingled with equally great perils. Navigating this future successfully, therefore, will require that states grapple seriously with the governance of AI risks.

Current approaches to AI risk governance focus on explicit standards and compliance mechanisms. As STS scholars have long maintained, however, such mechanisms are never sufficient. Extreme risks in complex systems can never be wholly captured by formal processes or governed by third parties. Technological safety in these circumstances hinges on complex, interpretive choices concerning everything from designs to deployment, and those choices are shaped more by economic incentives than by legislative oversight. Simply put, technological safety flourishes when it pays and struggles when it doesn’t.

While the impact of AIs on the economy has become a major focus for social scientists, however, little is said about the impact of the economy on AI development. We address this imbalance and argue that the risks of frontier AIs need to be understood in relation to their economisation. For all their novelty, AIs are built to be monetised, by organisations that exist to maximise capital; and attempts to understand, predict or steer their futures need to explore the incentive structures that arise from this.