Enhancing the reliability and accuracy of AI-enabled diagnosis via complementarity-driven deferral to clinicians (CoDoC)

Krishnamurthy Dvijotham, Jim Winkens, Melih Barsbey, Sumedh Ghaisas, Nick Pawlowski, Robert Stanforth, Patricia MacWilliams, Zahra S. Ahmed, Shekoofeh Azizi, Yoram Bachrach, Laura Culp, Mayank Daswani, Jan Freyberg, Christopher Kelly, Atilla P. Kiraly, Scott McKinney, Basil Mustafa, Vivek Natarajan, Krzysztof J. Geras, Jan Witowski, Zhi Zhen Qin, Jacob Creswell, Shravya Shetty, Marcin Sieniek, Terry Spitz, Greg S. Corrado, Pushmeet Kohli, Ali Taylan Cemgil, Alan KarthikesalingamResearch Square 2022 · DOI

Abstract

Diagnostic AI systems have achieved expert-level performance in medical imaging but can still fail in cases where clinicians succeed, and vice versa. We present the Complementarity-driven Deferral-to-Clinical Workflow (CoDoC) system, which learns to optimally select between AI and clinician decisions. CoDoC is compatible with various AI models and enhances accuracy in breast cancer and tuberculosis screening. Notably, it exceeded the "gold standard" of double-reading in a UK breast cancer screening program, reducing false positives by 25% and clinical workload by 66%.

Introduction: The Challenge of Human-AI Complementarity

While deep learning models often match or exceed human performance in specific diagnostic tasks, they possess different "failure modes" than human experts. An AI might be tripped up by rare image artifacts that a human easily dismisses, while a human might overlook subtle patterns that an AI identifies instantly. The goal of CoDoC is to leverage this complementarity by determining when to trust the AI and when to defer to a clinician.

Existing "learning to defer" methods often require retraining the diagnostic AI, which is impractical for "locked" medical devices regulated by health authorities. CoDoC addresses this by acting as a modular wrapper that optimizes the hand-off without altering the underlying AI model.

The CoDoC Framework

CoDoC operates by analyzing the confidence scores of a pre-existing diagnostic AI and comparing them against historical clinician performance. The system calculates an "advantage function" to decide if deferring to a clinician is likely to improve diagnostic outcomes (sensitivity or specificity).

The CoDoC deferral function, which decides whether to defer to a clinician based on the calculated advantage.

A key technical innovation in CoDoC is the use of Kernel Density Estimation (KDE) to smooth the advantage function. This approach mitigates "underspecification"—a common issue where multiple thresholds appear equally effective on small datasets but fail to generalize. By finding robust "zero-crossings" in the smoothed function, CoDoC ensures stable performance even when tuned on limited data (as few as 40 cases).

Experimental Validation and Results

The researchers evaluated CoDoC across diverse medical imaging modalities and clinical settings:

Breast Cancer Screening

  • UK Dataset: In a representative screening program (60,755 cases), CoDoC became the first system to outperform the "gold standard" of double-reading with arbitration. It achieved a 25% reduction in false positives while maintaining cancer detection rates and reducing clinician workload by 66%.
  • US Datasets: CoDoC exceeded the accuracy of single-reading by board-certified radiologists and two different state-of-the-art AI systems, demonstrating robustness across different manufacturers and populations.

Tuberculosis (TB) Screening

Using 23,954 chest X-rays from Bangladesh, CoDoC was tested against five commercially available AI systems. For three of these systems, CoDoC significantly improved specificity, reducing false positives by 5–15% while maintaining sensitivity.

Discussion and Future Directions

CoDoC demonstrates that the best diagnostic outcomes often come from a strategic partnership between humans and machines. By treating the AI as a "black box," CoDoC is highly adaptable to third-party tools and real-world clinical environments.

However, the study notes certain limitations. It assumes independent interpretations by clinicians and AI, whereas in practice, clinicians often use AI as an assistive tool, potentially introducing automation bias. Furthermore, while retrospective results are compelling, prospective clinical trials are necessary to validate how CoDoC performs in real-time human-AI interactions. To facilitate further research, the authors have open-sourced the CoDoC implementation.

Summary updated

Abstract

<title>Abstract</title> Diagnostic AI systems trained using deep learning have been shown to achieve expert-level identification of diseases in multiple medical imaging settings<sup>1,2</sup>. However, such systems are not always reliable and can fail in cases diagnosed accurately by clinicians and vice versa<sup>3</sup>. Mechanisms for leveraging this complementarity by learning to select optimally between discordant decisions of AIs and clinicians have remained largely unexplored in healthcare<sup>4</sup>, yet have the potential to achieve levels of performance that exceed that possible from either AI or clinician alone<sup>4</sup>. We develop a Complementarity-driven Deferral-to-Clinical Workflow (CoDoC) system that can learn to decide when to rely on a diagnostic AI model and when to defer to a clinician or their workflow. We show that our system is compatible with diagnostic AI models from multiple manufacturers, obtaining enhanced accuracy (sensitivity and/or specificity) relative to clinician-only or AI-only baselines in clinical workflows that screen for breast cancer or tuberculosis. For breast cancer, we demonstrate the first system that exceeds the accuracy of double-reading with arbitration (the “gold standard” of care) in a large representative UK screening program, with 25% reduction in false positives despite equivalent true-positive detection, while achieving a 66% reduction in clinical workload. In two separate US datasets, CoDoC exceeds the accuracy of single-reading by board certified radiologists and two different standalone state-of-the-art AI systems, with generalisation of this finding in different diagnostic AI manufacturers. For TB screening with chest X-rays, CoDoC improved specificity (while maintaining sensitivity) compared to standalone AI or clinicians for 3 of 5 commercially available diagnostic AI systems (5–15% reduction in false positives). Further, we show the limits of confidence score based deferral systems for medical AI, by demonstrating that no deferral strategy could have achieved significant improvement on the remaining two diagnostic AI systems. Our comprehensive assessment demonstrates that the superiority of CoDoC is sustained in multiple realistic stress tests for generalisation of medical AI tools along four axes: variation in the medical imaging modality; variation in clinical settings and human experts; different clinical deferral pathways within a given modality; and different AI softwares. Further, given the simplicity of CoDoC we believe that practitioners can easily adapt it and we provide an open-source implementation to encourage widespread further research and application.
BibTeX
@inproceedings{dvijotham2022enhancing,
  author = {Krishnamurthy Dvijotham and Jim Winkens and Melih Barsbey and Sumedh Ghaisas and Nick Pawlowski and Robert Stanforth and Patricia MacWilliams and Zahra S. Ahmed and Shekoofeh Azizi and Yoram Bachrach and Laura Culp and Mayank Daswani and Jan Freyberg and Christopher Kelly and Atilla P. Kiraly and Scott McKinney and Basil Mustafa and Vivek Natarajan and Krzysztof J. Geras and Jan Witowski and Zhi Zhen Qin and Jacob Creswell and Shravya Shetty and Marcin Sieniek and Terry Spitz and Greg S. Corrado and Pushmeet Kohli and Ali Taylan Cemgil and Alan Karthikesalingam},
  title = {{Enhancing the reliability and accuracy of AI-enabled diagnosis via complementarity-driven deferral to clinicians (CoDoC)}},
  booktitle = {Research Square},
  year = {2022},
  doi = {10.21203/rs.3.rs-2231672/v1},
}
Figure