TL;DR
This paper introduces a novel approach to medical automatic diagnosis (MAD) by adaptively aligning reinforcement learning (RL) for symptom inquiry with a classification model for diagnosis. They also present a new, more comprehensive dataset for simulating realistic patient interactions, achieving state-of-the-art accuracy with fewer diagnostic steps.
What Problem Does This Paper Address?
Medical automatic diagnosis (MAD) systems aim to mimic human doctors by interacting with patients to gather symptom information and ultimately diagnose a disease. While existing methods often treat symptom inquiry and diagnosis as separate tasks or try to unify them into a single complex problem, this paper highlights the limitations of these approaches. Specifically, symptom inquiry is a sequential decision-making problem where the goal is to efficiently gather informative symptoms, while diagnosis is a classification task. Forcing them into a single action space can increase complexity and reduce efficiency. Furthermore, existing datasets for training and evaluating MAD systems are often limited in the number of diseases and symptoms they cover, leading to unrealistic patient simulations. This paper addresses the challenge of effectively integrating symptom inquiry and diagnosis while also tackling the scarcity of high-quality, comprehensive datasets for MAD research.
Methods
The core of this paper's contribution lies in its novel method for aligning reinforcement learning (RL) and classification. The system comprises two main components:
- Symptom-Inquiring Agent (RL-based): This agent uses RL to decide which symptom to ask about next, aiming to maximize diagnostic accuracy while minimizing the number of inquiries. It's trained using a finite-horizon Markov Decision Process (MDP).
- Disease-Diagnosing Model (MLP Classifier): This is a multi-layer perceptron (MLP) that takes the current symptom information and outputs a probability distribution over possible diseases.
The key innovation is the adaptive alignment mechanism that bridges these two components. Instead of using a fixed stopping criterion, the paper proposes an adaptive threshold for deciding when to stop inquiring and make a diagnosis. This threshold is learned dynamically for each disease, adapting based on the disease classification model's confidence (measured by entropy). The system uses a composite reward function for the RL agent, combining a reward for correct diagnoses and penalties for incorrect ones or excessive inquiries, with an additional reward for discovering unknown symptoms. An entropy regularization term is also used to encourage exploration.
Datasets
The authors introduce a new dataset extracted from MedlinePlus, designed to overcome the limitations of existing benchmarks. This dataset is constructed from disease articles and includes:
- 893 diseases
- 1252 symptom-related concepts
- 860 medical examination-related concepts
This dataset is notable for including medical examination information, which is crucial for realistic diagnosis, and for its comprehensive coverage of diseases and symptoms compared to previous datasets like SymCat. They also used the SymCat dataset for comparison and evaluation, as is common in prior work.
Key Insights and Ideas
- Separation of Concerns is Key: Treating symptom inquiry (RL) and diagnosis (classification) as distinct but aligned tasks leads to better performance than unified approaches.
- Adaptive Thresholds are Crucial: A fixed stopping criterion is insufficient. Dynamically learning disease-specific thresholds for diagnosis based on model confidence (entropy) significantly improves accuracy and efficiency.
- Medical Examinations Matter: Including medical examinations as a type of "advanced symptom" in the dataset and model is vital for more realistic and accurate diagnoses.
- Negative Symptoms Provide Information: The entropy-based reward mechanism acknowledges that negative symptom feedback can be highly informative for ruling out diseases, not just positive symptoms.
- New Dataset Enables Better Evaluation: The proposed MedlinePlus dataset provides a more realistic and challenging benchmark for evaluating MAD systems.
Key Results
The proposed method achieved state-of-the-art results across multiple datasets, including their new MedlinePlus dataset.
- Higher Diagnosis Accuracy: The method consistently outperformed baselines like REFUEL, FIT, and INS-DS in terms of accuracy. For example, on the MedlinePlus dataset, it achieved 96.2% accuracy.
- Fewer Inquiring Turns: The system achieved these high accuracies with significantly fewer symptom inquiries compared to most baselines. On MedlinePlus, it required an average of 3.41 turns, far fewer than REFUEL (15.00) and comparable to INS-DS (1.33) but with much higher accuracy.
- Adaptive Thresholds Outperform Fixed Thresholds: Experiments showed that the adaptive threshold approach yielded higher diagnosis accuracy with fewer inquiring steps compared to using fixed thresholds, especially on larger datasets like MedlinePlus.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while their new dataset is more comprehensive, it's still based on simulated patients, not real-world clinical data.
- The performance of the adaptive threshold mechanism relies on the accuracy of the disease classifier.
- Future Work:
- The authors suggest exploring more sophisticated RL algorithms and network architectures.
- Further research could focus on incorporating more diverse patient information (e.g., medical history, demographics) into the model.
- The development of more realistic patient simulation environments and the use of real-world clinical data remain important future directions.
Summary updated