Human-machine partnership with artificial intelligence for chest radiograph diagnosis

Bhavik N. Patel, Louis Rosenberg, Gregg Willcox, David Baltaxe, Mimi Lyons, Jeremy Irvin, Pranav Rajpurkar, Timothy Amrhein, Rajan Gupta, Safwan Halabi, Curtis P. Langlotz, Edward Lo, Joseph Mammarappallil, A. J. Mariano, Geoffrey Riley, Jayne Seekins, Luyao Shen, Evan Zucker, Matthew P. Lungren

Abstract

Human-in-the-loop (HITL) AI may enable an ideal symbiosis of human experts and AI models, harnessing the advantages of both while at the same time overcoming their respective limitations. The purpose of this study was to investigate a novel collective intelligence technology designed to amplify the diagnostic accuracy of networked human groups by forming real-time systems modeled on biological swarms. Using small groups of radiologists, the swarm-based technology was applied to the diagnosis of pneumonia on chest radiographs and compared against human experts alone, as well as two state-of-the-art deep learning AI models. Our work demonstrates that both the swarm-based technology and deep-learning technology achieved superior diagnostic accuracy than the human experts alone. Our work further demonstrates that when used in combination, the swarm-based technology and deep-learning technology outperformed either method alone. The superior diagnostic accuracy of the combined HITL AI solution compared to radiologists and AI alone has broad implications for the surging clinical AI deployment and implementation strategies in future practice.
BibTeX
@article{patel2019human,
  author = {Bhavik N. Patel and Louis Rosenberg and Gregg Willcox and David Baltaxe and Mimi Lyons and Jeremy Irvin and Pranav Rajpurkar and Timothy Amrhein and Rajan Gupta and Safwan Halabi and Curtis P. Langlotz and Edward Lo and Joseph Mammarappallil and A. J. Mariano and Geoffrey Riley and Jayne Seekins and Luyao Shen and Evan Zucker and Matthew P. Lungren},
  title = {{Human-machine partnership with artificial intelligence for chest radiograph diagnosis}},
  journal = {Npj Digit. Medicine},
  year = {2019},
  doi = {10.1038/s41746-019-0189-7},
}
Figure