Introduction
Current AI risk assessment frameworks often treat risk as a universal constant, failing to account for the diverse perspectives of different stakeholders. A "risk" to a software developer might be seen as a "feature" by a business manager or a "catastrophe" by an end-user. This paper introduces a framework that leverages Large Language Models (LLMs) to simulate these varied perceptions, providing a more inclusive and transparent approach to AI governance.
Methodology: A Multi-Stage Pipeline
The researchers developed a pipeline to transform generic AI use cases into detailed maps of stakeholder conflicts. This process moves from high-level descriptions to granular, rule-based justifications.
The process involves five key stages:
- Persona Generation: Using GPT-4o, the system generates relevant stakeholders (e.g., surgeons, patients, administrators) for a given AI use case.
- Prompt Robustness: To ensure consistency, the model evaluates six linguistic variations of each scenario (e.g., semantic changes or word order) to verify that risk identification isn't a fluke of wording.
- Risk Inference: Utilizing the IBM Risk Atlas Nexus (RAN), scenarios are mapped against a comprehensive AI risk taxonomy.
- Policy Explanation (GloVE): The framework generates human-readable "IF-DESPITE" rules to explain risk judgments, such as: "IF reviewed by a medical professional DESPITE containing unreliable output."
- Conflict Scoring: Semantic similarity between justifications is measured to identify where stakeholders reason about the same concepts from opposing directions.
Experimental Setup
The framework was evaluated across three real-world domains involving high-stakes AI deployment:
- AI Medical Diagnosis Assistant: Determining if a patient requires surgery.
- Autonomous Vehicle System: Ensuring passengers reach destinations safely.
- AI Fraud Detection: Determining if customer transactions should be blocked.
Key Results and Insights
The study confirms that risk perception is deeply contextual. For instance, fraud analysts flagged significantly higher proportions of risks than other stakeholders, likely due to their professional focus on edge cases and system vulnerabilities.
The "Human Oversight" Paradox
A critical finding is the disagreement over human-in-the-loop safeguards. One stakeholder may view a system as safe because of human oversight (the "IF" clause), while another views it as risky despite that oversight, seeing the human as a potential failure point or an insufficient safeguard.
Visualizing Conflict
The authors developed an interactive visualization tool to help human governors navigate these frictions.
The tool uses color-coded bubbles where size indicates the number of conflicts a stakeholder is involved in. Users can drill down into specific "IF-DESPITE" rules to understand the root cause of the friction. Quantitatively, the AI Fraud Detection use case showed the highest conflict rate (66%), while Medical Diagnosis had the lowest (21.27%), indicating a higher degree of consensus in clinical settings.
Conclusion and Future Work
While the use of synthetic personas and binary risk classifications presents limitations, this framework provides a scalable method for identifying stakeholder friction before AI systems are deployed. Future work aims to integrate real human feedback to validate persona accuracy and expand the framework to support the gradient risk levels (e.g., Low, Medium, High) required by regulations like the EU AI Act.
Summary updated