AI-Driven Diagnostic Assistance in Medical Inquiry: Reinforcement Learning Algorithm Development and Validation

Xuan Zou, Weijie He, Yu Huang, Yi Ouyang, Zhen Zhang, Yu Wu, Yongsheng Wu, Lili Feng, Sheng Wu, Mengqi Yang, Xuyan Chen, Yefeng Zheng, Rui Jiang, Ting ChenJournal of Medical Internet Research 2024 · DOI

TL;DR

This paper introduces MedRIA, an AI system that acts as a diagnostic assistant by intelligently guiding medical inquiries using reinforcement learning. MedRIA simulates a clinician's diagnostic process, achieving high accuracy and consistency with human doctors, and significantly reducing the number of questions needed for diagnosis.

What Problem Does This Paper Address?

The traditional medical diagnostic process is complex and often relies on a clinician's experience, which can be prone to biases and information gaps. Existing AI diagnostic tools often focus on providing a diagnosis at the end of the process, neglecting the crucial information-gathering phase. This gap is particularly problematic in resource-constrained settings where timely and accurate diagnoses are critical. The paper aims to bridge this gap by developing an AI that can assist clinicians in the inquiry phase, mimicking and optimizing the diagnostic investigation logic.

Methods

The core of MedRIA is a reinforcement learning actor-critic framework.

  • Feature Extraction: Electronic Health Records (EHRs) are processed using natural language processing (NLP) and a pre-trained model called MedBERT to extract relevant features from clinical text and structured lab results.
  • Reinforcement Learning: An actor-critic model is trained to learn an optimal inquiry strategy. The "actor" decides which question to ask next, while the "critic" evaluates the quality of the actions.
  • Inquiry Stages: The AI's inquiry process is structured into four stages: symptoms/medical history, physical examination, auxiliary examinations, and termination with a diagnosis.
  • VAE Integration: A Variational Autoencoder (VAE) is used within the actor to model conditional probabilities between observed and unobserved patient features, helping to predict what information is most valuable to inquire about.
  • Reward Shaping: A carefully designed reward function guides the learning process, penalizing inefficient inquiries and rewarding accurate diagnostic outcomes.

Datasets

The study utilized multicenter, deidentified outpatient Electronic Health Records (EHRs) from 76 hospitals in Shenzhen, China, collected between July and November 2021.

  • Emergency Department Data: 339,020 patient cases.
  • Pediatrics Department Data: 561,659 patient cases. These datasets were split into training, validation, and testing sets. For external validation, the IMCS-21 medical dialogue corpus was used, containing 4116 online medical consultations.

Key Insights and Ideas

  • Focus on the Inquiry Process: The paper highlights the critical, yet often overlooked, importance of the information-gathering phase in diagnosis and proposes an AI solution specifically for this stage.
  • Reinforcement Learning for Dynamic Decision-Making: Using RL allows the AI to learn a dynamic and adaptive inquiry strategy, rather than a static set of rules, which is crucial for the complex and evolving nature of medical diagnosis.
  • Simulating Clinician Logic: The AI's inquiry process is designed to mimic the sequential and logical steps a human clinician would take, making it more intuitive and trustworthy for medical professionals.
  • AI as a Collaborative Tool: The research demonstrates that AI can effectively collaborate with physicians, not just replace them, by suggesting relevant questions and reducing the overall inquiry burden.
  • Potential for Resource-Limited Settings: The AI's ability to optimize inquiry and potentially reduce the need for extensive questioning makes it a promising tool for areas with limited medical resources or expertise.

Key Results

  • High Diagnostic Performance: MedRIA achieved high diagnostic performance, with AUROCs of 0.955 and 0.943 in the emergency and pediatrics tasks, respectively, when acting alone.
  • Reduced Inquiry Load: In simulated collaboration with physicians, MedRIA reduced the average number of physician inquiries by 54% (to 46% of the original number) in the emergency task and 57% (to 43% of the original number) in the pediatrics task.
  • Inquiry Consistency: External validation showed that MedRIA's inquiry logic was consistent with physicians, with a normalized Kendall τ distance of 0.323, indicating reasonable alignment.
  • Comparable to Physicians: In collaborative scenarios, MedRIA helped achieve diagnostic metrics (AUROC, F1-score, accuracy) comparable to, and in some cases exceeding, those of physicians. For instance, collaborative inquiry achieved AUROCs of 0.972 and 0.968 in the emergency and pediatrics tasks, respectively.

Limitations and Future Work

  • Feature Engineering: The paper acknowledges that EHR feature extraction requires careful engineering, which could be further refined.
  • Missed Inquiries: MedRIA is not immune to missing inquiries, particularly regarding auxiliary examinations (AEs). Future work could involve modeling the significance of inquiries for disease treatment, not just diagnosis.
  • Data Bias and Generalizability: The study's data was heavily skewed towards upper respiratory tract infections due to seasonal variations and potentially limited access to primary care in China. Further validation in different countries and with different disease distributions is needed. The current evaluation is limited to Chinese text.
  • Over-Certainty Bias: The AI might exhibit over-certainty. Future work should focus on managing clinical risks by incorporating confidence levels and developing safe integration systems.
  • LLM Integration: The authors suggest incorporating MedRIA with the latest developments in Large Language Models (LLMs) for future deployment and testing.

Summary updated

Abstract

This retrospective analysis of predominantly respiratory pediatric presentations in emergency and pediatrics departments demonstrated that an AI-driven diagnostic assistant had high diagnostic performance both in stand-alone use and in simulated collaboration with clinicians. Its investigation process was found to be consistent with the clinicians' medical investigation logic. These findings highlight the diagnostic assistant's promise in assisting the decision-making processes of health care professionals.
BibTeX
@article{zou2024ai,
  author = {Xuan Zou and Weijie He and Yu Huang and Yi Ouyang and Zhen Zhang and Yu Wu and Yongsheng Wu and Lili Feng and Sheng Wu and Mengqi Yang and Xuyan Chen and Yefeng Zheng and Rui Jiang and Ting Chen},
  title = {{AI-Driven Diagnostic Assistance in Medical Inquiry: Reinforcement Learning Algorithm Development and Validation}},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/54616},
}
Figure