TL;DR
This paper argues that the current focus on existential risks in AI safety research is too narrow and risks alienating important contributors. Through a systematic literature review, they demonstrate that AI safety research already encompasses a vast array of practical, near-term concerns, mirroring traditional engineering safety practices. They advocate for a more inclusive definition of AI safety that embraces diverse motivations and perspectives.
What Problem Does This Paper Address?
The discourse around AI safety has increasingly become synonymous with mitigating existential risks from advanced AI systems. While this focus is important, the authors argue it has three drawbacks: it may exclude researchers with different motivations, it can lead the public to believe AI safety is solely about doomsday scenarios, and it might create resistance from those who are skeptical of existential risk predictions. This narrow framing overlooks decades of work in technological and systems safety that are directly applicable to AI. The paper aims to contextualize AI safety within this broader engineering tradition and highlight the diverse range of safety concerns already being addressed.
Methods
The authors conducted a systematic literature review of primarily peer-reviewed AI safety research. They employed a multi-stage query process on Web of Science and Scopus databases, using a hierarchical query structure to identify relevant papers. This was complemented by snowball sampling to capture emerging research. The review focused on two key research questions: (1) What types of risks are addressed across the AI system lifecycle? and (2) What mitigation strategies are proposed? Papers were then annotated and categorized based on their methodological approach, the risks they addressed, and the methods used for mitigation.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- AI Safety is Already Broad: The review reveals a vast landscape of concrete AI safety work addressing immediate and practical concerns, not just future existential threats.
- Connection to Traditional Engineering: AI safety research naturally extends existing technological and systems safety concerns and practices, drawing parallels with fields like aviation and pharmaceutical safety.
- Epistemic Inclusivity is Key: A more inclusive and pluralistic conception of AI safety is needed to accommodate the full range of safety considerations, motivations, and perspectives.
- Dichotomy is Oversimplified: The common distinction between near-term and long-term (existential) AI safety challenges may oversimplify the rich and interconnected nature of the research.
- Shared Vocabulary and Challenges: Concepts like corrigibility and adversarial robustness span multiple time horizons, suggesting a more integrated approach is beneficial.
Key Results
The systematic review identified eight overarching risk types addressed in AI safety research, with "Noise and outliers" being the most frequent (23.2%), followed by "Lack of monitoring" (15.2%) and "System misspecification" (13.9%). Other significant risks include "Lack of control enforcement" (12.0%), "Unsafe exploration" (11.4%), "Adversarial attack" (10.3%), "Non-stationary distributions" (9.9%), and "Undesirable behavior" (4.2%). The research landscape is dynamic, with a significant increase in publications since 2016. Methodologically, the research is divided between theoretical work and applied/evaluated work, with applied research often focusing on concrete algorithms and theoretical work on general frameworks.
Limitations and Future Work
The authors acknowledge several limitations:
- Dynamic Field: The review captures a snapshot in time and may not fully reflect the most recent advancements.
- Exclusion of Pre-prints: The focus on peer-reviewed research excludes a significant portion of AI safety discourse found in pre-print repositories like arXiv.
- Annotator Bias: While efforts were made to mitigate bias, the annotation process inherently involves some degree of subjectivity.
- Exclusion of Online Forums: Discussions on platforms like LessWrong and the AI Alignment Forum were not included.
Future work suggested includes:
- Sociotechnical AI Safety: Further investigation into the concept of "sociotechnical AI safety" and its implications for research and practice, integrating social, ethical, and political considerations.
- Broader Source Inclusion: Expanding future research to include non-peer-reviewed sources (pre-prints, technical reports, online forums) for a more comprehensive understanding.
Summary updated