Towards a New Participatory Approach for Designing Artificial Intelligence and Data-Driven Technologies

Soaad Hossain, Syed Ishtiaque AhmedarXiv 2021 · arXiv

TL;DR

This paper proposes "agile participatory design" (APD), a novel approach to designing AI and data-driven technologies that aims to better include marginalized communities. APD addresses limitations in traditional participatory design by emphasizing broader alliances and more integrated involvement throughout the AI development lifecycle, aiming for more equitable and impactful technology.

What Problem Does This Paper Address?

The increasing prevalence of AI and data-driven technologies raises significant ethical and practical concerns, particularly regarding their impact on marginalized communities. These communities are often excluded from the design and development process, leading to technologies that may not serve their needs or could even exacerbate existing inequalities. While participatory design (PD) has been a valuable approach for involving marginalized groups in the design of social development and IT solutions, its current application faces challenges when dealing with the complexities and rapid evolution of AI technologies. Specifically, the opacity of AI models, the dynamic nature of machine learning, and the difficulty in engaging users within the typical AI development lifecycle pose significant hurdles. This paper seeks to bridge this gap by proposing a refined PD methodology tailored for AI.

Methods

The paper introduces agile participatory design (APD), a new approach that builds upon existing PD principles but adapts them for AI and data-driven technologies. APD is structured around three key components:

  • Participants: APD emphasizes the inclusion of activists from marginalized communities and/or marginalized community members themselves. It also advocates for "engaged partners" such as ethicists, public health experts, legal practitioners, and social workers who are invested in the well-being of individuals and the public. This broader participant base aims to focus on larger societal issues and drive significant, lasting change.
  • Alliances: To overcome the challenge of sustaining engagement and ensuring the deployed AI system meets the needs of all stakeholders, APD proposes forming alliances. These alliances would involve management, users, and other stakeholders, ensuring that the AI system aligns with the needs of marginalized communities and activists, and helping to resist negative design changes.
  • Involvement: APD integrates principles from agile software development, suggesting that users and stakeholders should be involved across multiple phases of the AI system lifecycle. This continuous involvement is crucial for ensuring the AI technology meets user and stakeholder expectations. The paper also notes that for confidentiality, datasets may need to be anonymized, with only key demographic information shared.

The paper also outlines a four-phase model for AI system design (Objectives, Data Collection, Model Development, Model Application) and illustrates how APD can be integrated into this process, including management approval of the final machine learning model.

Datasets

Not discussed in the paper. The paper focuses on the methodology of participatory design for AI rather than empirical evaluation using specific datasets.

Key Insights and Ideas

  • AI's unique challenges for PD: The inherent complexity and "black box" nature of many AI technologies (especially those using neural networks or complex algorithms like SVMs) make traditional PD difficult, as designers and users may struggle to understand how decisions are made.
  • The need for broader alliances: Simply involving users in the design phase isn't enough. Building strong alliances between management, users, and other stakeholders is crucial for the successful deployment and long-term impact of AI technologies, especially in addressing systemic issues.
  • Agile principles for PD in AI: Adapting agile software development methodologies, such as continuous involvement and iterative feedback, can make PD more effective for the dynamic nature of AI development.
  • Beyond product-level design: Current PD often focuses on improving existing products or small-scale solutions. APD aims to shift this focus towards designing AI technologies that can enact profound, positive changes in the lives of marginalized communities.
  • The role of activists: Recognizing that direct participation from marginalized community members can be challenging due to various barriers, the paper highlights the importance of involving trusted activists who can effectively advocate for community needs.

Key Results

This paper is primarily theoretical and proposes a new methodology. Therefore, it does not present quantitative results from empirical studies. Its key "results" are the conceptual framework and arguments for the proposed agile participatory design approach.

Limitations and Future Work

The authors acknowledge that APD is not a perfect solution and that there are likely aspects that could be further incorporated. They suggest:

  • Further investigation and modification: APD needs to be further explored, modified, and expanded to cover additional important issues not addressed in the paper.
  • Testing and validation: Empirical testing of APD is necessary to obtain data that can validate the design method.
  • Algorithm selection involvement: The paper briefly mentions that users and stakeholders might ideally be involved in the machine learning algorithm selection process, but this is not fully elaborated upon.
  • Addressing specific AI challenges: While the paper discusses challenges like machine learning comprehension and AI system training/application, further work is needed to detail how APD specifically mitigates these in practice.

Summary updated

BibTeX
@misc{hossain2021towards,
  author = {Soaad Hossain and Syed Ishtiaque Ahmed},
  title = {{Towards a New Participatory Approach for Designing Artificial Intelligence and Data-Driven Technologies}},
  year = {2021},
  eprint = {2104.04072},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.04072},
}
Figure