Robust diagnostic classification via Q-learning

Victor Ardulov, Victor R. Martínez, Krishna Somandepalli, Shuting Zheng, Emma Salzman, Catherine Lord, Somer Bishop, Shrikanth NarayananScientific 2021 · DOI

Abstract

Machine learning (ML) models have demonstrated the power of utilizing clinical instruments to provide tools for domain experts in gaining additional insights toward complex clinical diagnoses. In this context these tools desire two additional properties: interpretability, being able to audit and understand the decision function, and robustness, being able to assign the correct label in spite of missing or noisy inputs. This work formulates diagnostic classification as a decision-making process and utilizes Q-learning to build classifiers that meet the aforementioned desired criteria. As an exemplary task, we simulate the process of differentiating Autism Spectrum Disorder from Attention Deficit-Hyperactivity Disorder in verbal school aged children. This application highlights how reinforcement learning frameworks can be utilized to train more robust classifiers by jointly learning to maximize diagnostic accuracy while minimizing the amount of information required.

Introduction

In clinical settings, diagnosis is rarely a one-shot event where all necessary data is available simultaneously. Instead, it is a progressive, multi-step inquiry. However, most machine learning (ML) models used in healthcare assume a complete feature set is present at inference time. When faced with missing or noisy data—common in complex neurodevelopmental disorders—standard models like Decision Trees often fail. This paper addresses the challenge of differentiating Autism Spectrum Disorder (ASD) from Attention Deficit-Hyperactivity Disorder (ADHD) in children, conditions with overlapping behavioral presentations. The objective is to build a tool that is both interpretable for clinicians and robust to missing information.

Methods: Diagnosis as a Sequential Process

The researchers reformulate clinical diagnosis as a Markov Decision Process (MDP) solved using Q-learning. In this framework, an agent acts as the clinician, choosing to either "Inquire" about a specific symptom (from the ADI-R interview) or "Predict" the final diagnosis.

The diagnostic process modeled as a sequential inquiry session.

The "reward function" is central to this approach: the agent receives a high reward for a correct diagnosis but is penalized for every question asked. This forces the model to find the most efficient path to a confident conclusion. A Naive Bayes classifier provides the "belief state" (the probability of ASD vs. ADHD) based on gathered information. Unlike static decision trees, the Q-learning policy learns that the value of information depends on the current context, allowing for a more holistic correlation of items.

Datasets and Clinical Samples

The study utilized a dataset of 463 children (344 with ASD, 119 with ADHD) who were clinically referred and assessed. Features were selected from the 10 most discriminative items of the Autism Diagnostic Interview-Revised (ADI-R), covering behaviors such as reciprocal conversation and social smiling.

Distribution of age, Full Scale IQ (FSIQ), and Verbal IQ (VIQ) across ASD and ADHD samples.

To ensure the model learned behavioral differences rather than developmental proxies, the groups were balanced for Age, Full Scale IQ (FSIQ), and Verbal IQ (VIQ).

Results and Robustness

The Q-learning policy was compared against standard Decision Trees (DT) and Random Forests (RF).

  • Full Data Performance: The Q-learning policy achieved an F1-score of 0.5620, outperforming the best Random Forest (0.5064).
  • Graceful Degradation: The most significant finding is the model's performance under extreme data loss. When 9 out of 10 items were missing, the Q-learning model remained stable with an F1-score of 0.4312. In contrast, standard Decision Trees plummeted to 0.0307, and even "robust" DTs (trained with data masking) only managed 0.1211.

The policy remains interpretable, as it can be visualized as a transparent decision flow where clinicians can see the "belief" shift after each response.

Conclusion and Future Directions

While the current Q-table approach is limited by state-space explosion (restricting it to 10 features), it demonstrates a powerful proof-of-concept for robust diagnosis. Future work involves utilizing Deep Q-Networks (DQN) to handle larger clinical instruments with hundreds of items. Additionally, "human-in-the-loop" systems could allow the AI to suggest the next best question to a clinician in real-time, integrating spontaneous clinical observations into the decision-making process.

Summary updated

Abstract

Machine learning (ML) models have demonstrated the power of utilizing clinical instruments to provide tools for domain experts in gaining additional insights toward complex clinical diagnoses. In this context these tools desire two additional properties: interpretability, being able to audit and understand the decision function, and robustness, being able to assign the correct label in spite of missing or noisy inputs. This work formulates diagnostic classification as a decision-making process and utilizes Q-learning to build classifiers that meet the aforementioned desired criteria. As an exemplary task, we simulate the process of differentiating Autism Spectrum Disorder from Attention Deficit-Hyperactivity Disorder in verbal school aged children. This application highlights how reinforcement learning frameworks can be utilized to train more robust classifiers by jointly learning to maximize diagnostic accuracy while minimizing the amount of information required.
BibTeX
@inproceedings{ardulov2021robust,
  author = {Victor Ardulov and Victor R. Martínez and Krishna Somandepalli and Shuting Zheng and Emma Salzman and Catherine Lord and Somer Bishop and Shrikanth Narayanan},
  title = {{Robust diagnostic classification via Q-learning}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-90000-4},
}
Figure