Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice

Emma Kallina, Thomas Bohné, Jatinder Singh

TL;DR

Responsible AI (rAI) guidance strongly advocates for stakeholder involvement (SHI) to ensure ethical AI development. However, this study reveals a significant disconnect: current industry SHI practices, driven by commercial priorities like customer value and compliance, largely fail to achieve the broader societal goals promoted by rAI guidance. This misalignment hinders the development of truly responsible AI.

What Problem Does This Paper Address?

The field of Responsible AI (rAI) is increasingly emphasizing the importance of Stakeholder Involvement (SHI) throughout the AI development lifecycle. This is seen as crucial for addressing potential harms and ensuring AI systems align with societal values. However, SHI is also a long-standing practice in commercial software development, often with different motivations, primarily focused on product success and market fit. The core problem this paper addresses is understanding the extent to which these established, commercially-driven SHI practices in the AI industry actually contribute to the goals of rAI guidance, and identifying the disconnects that prevent this alignment. Without this understanding, efforts to promote responsible AI through SHI may be ineffective.

Methods

The researchers employed a mixed-methods approach to tackle their research questions:

  1. Analysis of rAI Guidance Documents: They analyzed 56 rAI guidance documents from 29 organizations (multi-stakeholder, intergovernmental, civil society) to identify the stated benefits and goals associated with SHI in rAI development. This involved thematic analysis of the documents.
  2. Industry Practice Investigation: To understand current SHI practices, they conducted:
  3. An online survey with 130 AI practitioners to gather broad insights into their experiences, drivers, and methods for SHI.
  4. Semi-structured interviews with 10 AI practitioners to gain deeper qualitative insights into the mechanisms and challenges behind these practices.

The findings from these two phases were then compared to address the central question of alignment between guidance and practice.

Datasets

Not discussed in the paper. The study analyzed existing rAI guidance documents and collected primary data through surveys and interviews with AI practitioners.

Key Insights and Ideas

  • Guidance vs. Reality: rAI guidance promotes SHI for goals like redistributing power, improving socio-technical understanding, anticipating risks, and enabling public oversight. In contrast, industry practice is overwhelmingly driven by commercial interests (customer value, usability) and compliance.
  • Commercial Priorities Dominate: The primary motivations for SHI in AI development are to increase customer value, ensure product-market fit, and meet legal or reputational compliance requirements.
  • Limited Scope of Involvement: Current SHI practices tend to involve stakeholders directly tied to commercial interests (e.g., end-users, domain experts) and often neglect marginalized or at-risk communities, even when they are significantly impacted.
  • Low Stakeholder Agency: SHI in practice often grants stakeholders very limited decision-making power, typically involving them in later stages for feedback or testing rather than co-designing objectives or core features.
  • Conflicts Discourage rAI-Aligned SHI: Tensions arise between commercial interests and the goals of rAI, as well as conflicts with management's or colleagues' personal beliefs, which actively discourage SHI practices that go beyond traditional commercial aims.
  • "Ethics Washing" Risk: The disconnect suggests a risk of "ethics washing," where organizations may claim to engage in SHI without genuinely addressing the broader societal implications promoted by rAI guidance.

Key Results

  • Five Key Benefits of SHI in rAI Guidance: The analysis of guidance documents identified five core benefits: rebalancing decision power, detailed understanding of the socio-technical context, improved risk anticipation, increased public understanding and trust, and enabling public scrutiny and monitoring.
  • Commercial Drivers for SHI: 58% of survey participants cited "understanding stakeholder needs" and 52% "understanding stakeholder concerns" as key drivers, aligning with traditional SHI. Commercial interests were the most frequent motivations.
  • Compliance as a Driver: 28% of participants cited "to meet legal requirements" as a driver, highlighting the importance of compliance.
  • Limited Involvement of Broader Public: Only 11% of participants involved the general public, despite 28% acknowledging their system affects this group. Similarly, only 11% involved affected non-users, despite 32% reporting them being affected.
  • Late-Stage Involvement: External stakeholders were primarily engaged through prototype (42%) or usability testing (41%), typically at later stages of development.
  • Low Contribution to rAI Goals: The study estimates that current SHI practices contribute "Very Low" to rebalancing decision power and enabling public scrutiny, and "Medium with Limited Scope" to understanding socio-technical context and improved risk anticipation.

Limitations and Future Work

The authors acknowledge several limitations:

  • Sample Bias: The survey participants might be skewed towards those already interested in SHI, potentially underestimating the lack of SHI in less engaged practitioners.
  • Interviewee Availability: The small number of interviewees (n=10) reflects the difficulty of recruiting expert AI practitioners for extended periods.
  • Demographic Scope: The expert sample was predominantly male and UK-based, which may not be representative of the broader AI practitioner workforce.

Future work suggested includes:

  • Co-creating Clearer Guidance: Developing guidance that explicitly states the early onset and continuity of SHI, the shift in decision-making power, and the scope of involvement needed for rAI.
  • Clarifying Terminology: Establishing more specific terms for different types of SHI (e.g., participatory development, public participation, expert involvement, public oversight) to avoid confusion and "ethics washing."
  • Leveraging Regulation: Exploring how legal pressure and incentives can drive more rAI-oriented SHI, particularly by tying it more directly to commercial interests and addressing conflicts.
  • Developing Actionable Tools: Creating more actionable tools and methods to support SHI practices, especially for addressing conflicting stakeholder needs and commercial interests.
  • Research Agenda: The paper proposes a detailed research agenda covering understanding SHI across the AI lifecycle, designing improved guidance and tools, and informing AI regulation and public participation.

Summary updated

Abstract

Responsible AI (rAI) guidance increasingly promotes stakeholder involvement (SHI) during AI development.At the same time, SHI is already common in commercial software development, but with potentially different foci.This study clarifies the extent to which established SHI practices are able to contribute to rAI efforts as well as potential disconnects -essential insights to inform and tailor future interventions that further shift industry practice towards rAI efforts.First, we analysed 56 rAI guidance documents to identify why SHI is recommended (i.e. its expected benefits for rAI) and uncovered goals such as redistributing power, improving socio-technical understandings, anticipating risks, and enhancing public oversight.To understand why and how SHI is currently practised in commercial settings, we then conducted an online survey (n=130) and semi-structured interviews (n=10) with AI practitioners.Our findings reveal that SHI in practice is primarily driven by commercial priorities (e.g.customer value, compliance) and several factors currently discourage more rAI-aligned SHI practices.This suggests that established SHI practices are largely not contributing to rAI efforts.Towards addressing this disconnect, we propose interventions and research opportunities to advance SHI for rAI development in real-world practice.
BibTeX
@misc{kallina2025stakeholder,
  author = {Emma Kallina and Thomas Bohné and Jatinder Singh},
  title = {{Stakeholder Participation for Responsible AI Development: Disconnects Between Guidance and Current Practice}},
  year = {2025},
  eprint = {2506.09873},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.09873},
}
Figure