Toward fairness in artificial intelligence for medical image analysis: identification and mitigation of potential biases in the roadmap from data collection to model deployment

Karen Drukker, Weijie Chen, Judy Wawira Gichoya, Nicholas P. Gruszauskas, Jayashree Kalpathy–Cramer, Oluwasanmi Koyejo, Kyle J. Myers, Rui Carlos Sá, Berkman Sahiner, Heather M. Whitney, Zi Zhang, Maryellen L. GigerJournal of Medical Imaging 2023 · DOI

TL;DR

This paper provides a comprehensive roadmap for identifying and mitigating biases in medical imaging AI, covering the entire pipeline from data collection to model deployment. It's crucial for ensuring fair and trustworthy AI in healthcare, preventing the exacerbation of existing inequities.

What Problem Does This Paper Address?

Artificial intelligence (AI) and machine learning (ML) are rapidly transforming medical imaging, offering powerful tools for disease detection, diagnosis, and prognosis. However, these AI systems are susceptible to biases introduced at various stages of their development and deployment. These biases can lead to systematic differences in how AI models perform for different patient groups, potentially exacerbating existing health inequities. The paper addresses the critical need to understand, identify, and mitigate these biases to ensure that medical imaging AI is fair, trustworthy, and equitable for all.

Methods

The authors, a multi-institutional team of experts, took a systematic approach to identify and categorize biases in medical imaging AI. They mapped out the AI/ML pipeline into five key stages: data collection, data preparation and annotation, model development, model evaluation, and model deployment. Within each stage, they identified specific sources of potential bias and proposed mitigation strategies. This comprehensive framework allows for a structured understanding of where and how biases can emerge and be addressed.

Datasets

The paper does not focus on a specific dataset for training or evaluation. Instead, it discusses the general challenges and considerations related to data collection and preparation that can introduce bias. It references the Medical Imaging Data and Resource Center (MIDRC) as a significant effort to create a curated, representative imaging data commons to foster fair and equitable AI development.

Key Insights and Ideas

  • Bias is a Pipeline Problem: Bias isn't confined to a single stage; it can be introduced and propagated throughout the entire AI development lifecycle, from initial data collection to final deployment.
  • Five Stages of Bias: The paper clearly delineates five critical stages where bias can manifest: data collection, preparation/annotation, model development, evaluation, and deployment.
  • Interconnectedness of Biases: Many biases are not isolated. For instance, population bias in data collection can lead to issues in model evaluation and deployment.
  • Beyond Technical Fixes: While technical solutions are important, the paper emphasizes the need for diverse teams, careful documentation, transparency, and a deep understanding of the clinical context to address bias effectively.
  • "Ground Truth" is Not Always True: The concept of a "reference standard" or "ground truth" can itself be biased, leading to AI models that learn and perpetuate these inaccuracies.
  • Federated Learning Isn't a Silver Bullet: While federated learning offers potential benefits for privacy and data diversity, it doesn't automatically eliminate all biases and can introduce new challenges.

Key Results

The paper identifies 29 distinct sources of potential bias across the five stages of the AI/ML pipeline. For each bias, it provides a definition and suggests potential mitigation strategies. The key "result" is the comprehensive framework and detailed catalog of biases, serving as a valuable resource for researchers and clinicians. No specific quantitative performance metrics are presented, as the paper's focus is on the identification and categorization of biases.

Limitations and Future Work

The authors acknowledge that their list of biases is extensive and that some biases can be categorized in multiple ways. They also note that while federated learning has potential, it doesn't solve all bias issues and can introduce its own set of challenges. Future work includes translating this research into an interactive online tool and continuing to focus on fairness and equity in AI development.

Summary updated

Abstract

Our findings provide a valuable resource to researchers, clinicians, and the public at large.
BibTeX
@article{drukker2023toward,
  author = {Karen Drukker and Weijie Chen and Judy Wawira Gichoya and Nicholas P. Gruszauskas and Jayashree Kalpathy–Cramer and Oluwasanmi Koyejo and Kyle J. Myers and Rui Carlos Sá and Berkman Sahiner and Heather M. Whitney and Zi Zhang and Maryellen L. Giger},
  title = {{Toward fairness in artificial intelligence for medical image analysis: identification and mitigation of potential biases in the roadmap from data collection to model deployment}},
  journal = {Journal of Medical Imaging},
  year = {2023},
  doi = {10.1117/1.jmi.10.6.061104},
}
Figure