Who Sees the Risk? Stakeholder Conflicts and Explanatory Policies in LLM-based Risk Assessment

Srishti Yadav, Jasmina Gajcin, Erik Miehling, Elizabeth DalyarXiv 2025 · DOI · arXiv

Introduction

Current AI risk assessment frameworks often treat risk as a universal constant, failing to account for the diverse perspectives of different stakeholders. A "risk" to a software developer might be seen as a "feature" by a business manager or a "catastrophe" by an end-user. This paper introduces a framework that leverages Large Language Models (LLMs) to simulate these varied perceptions, providing a more inclusive and transparent approach to AI governance.

Methodology: A Multi-Stage Pipeline

The researchers developed a pipeline to transform generic AI use cases into detailed maps of stakeholder conflicts. This process moves from high-level descriptions to granular, rule-based justifications.

The multi-stage pipeline for stakeholder-grounded risk assessment.

The process involves five key stages:

  1. Persona Generation: Using GPT-4o, the system generates relevant stakeholders (e.g., surgeons, patients, administrators) for a given AI use case.
  2. Prompt Robustness: To ensure consistency, the model evaluates six linguistic variations of each scenario (e.g., semantic changes or word order) to verify that risk identification isn't a fluke of wording.
  3. Risk Inference: Utilizing the IBM Risk Atlas Nexus (RAN), scenarios are mapped against a comprehensive AI risk taxonomy.
  4. Policy Explanation (GloVE): The framework generates human-readable "IF-DESPITE" rules to explain risk judgments, such as: "IF reviewed by a medical professional DESPITE containing unreliable output."
  5. Conflict Scoring: Semantic similarity between justifications is measured to identify where stakeholders reason about the same concepts from opposing directions.

Experimental Setup

The framework was evaluated across three real-world domains involving high-stakes AI deployment:

  • AI Medical Diagnosis Assistant: Determining if a patient requires surgery.
  • Autonomous Vehicle System: Ensuring passengers reach destinations safely.
  • AI Fraud Detection: Determining if customer transactions should be blocked.

Key Results and Insights

The study confirms that risk perception is deeply contextual. For instance, fraud analysts flagged significantly higher proportions of risks than other stakeholders, likely due to their professional focus on edge cases and system vulnerabilities.

The "Human Oversight" Paradox

A critical finding is the disagreement over human-in-the-loop safeguards. One stakeholder may view a system as safe because of human oversight (the "IF" clause), while another views it as risky despite that oversight, seeing the human as a potential failure point or an insufficient safeguard.

Visualizing Conflict

The authors developed an interactive visualization tool to help human governors navigate these frictions.

The Stakeholder Conflict Visualization tool showing conflicting perspectives on AI risks.

The tool uses color-coded bubbles where size indicates the number of conflicts a stakeholder is involved in. Users can drill down into specific "IF-DESPITE" rules to understand the root cause of the friction. Quantitatively, the AI Fraud Detection use case showed the highest conflict rate (66%), while Medical Diagnosis had the lowest (21.27%), indicating a higher degree of consensus in clinical settings.

Conclusion and Future Work

While the use of synthetic personas and binary risk classifications presents limitations, this framework provides a scalable method for identifying stakeholder friction before AI systems are deployed. Future work aims to integrate real human feedback to validate persona accuracy and expand the framework to support the gradient risk levels (e.g., Low, Medium, High) required by regulations like the EU AI Act.

Summary updated

Abstract

Understanding how different stakeholders perceive risks in AI systems is essential for their responsible deployment. This paper presents a framework for stakeholder-grounded risk assessment by using LLMs, acting as judges to predict and explain risks. Using the Risk Atlas Nexus and GloVE explanation method, our framework generates stakeholder-specific, interpretable policies that shows how different stakeholders agree or disagree about the same risks. We demonstrate our method using three real-world AI use cases of medical AI, autonomous vehicles, and fraud detection domain. We further propose an interactive visualization that reveals how and why conflicts emerge across stakeholder perspectives, enhancing transparency in conflict reasoning. Our results show that stakeholder perspectives significantly influence risk perception and conflict patterns. Our work emphasizes the importance of these stakeholder-aware explanations needed to make LLM-based evaluations more transparent, interpretable, and aligned with human-centered AI governance goals.
BibTeX
@misc{yadav2025who,
  author = {Srishti Yadav and Jasmina Gajcin and Erik Miehling and Elizabeth Daly},
  title = {{Who Sees the Risk? Stakeholder Conflicts and Explanatory Policies in LLM-based Risk Assessment}},
  year = {2025},
  eprint = {2511.03152},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.03152},
}
Figure