Practical Strategies for Extreme Missing Data Imputation in Dementia Diagnosis

Niamh McCombe, Shuo Liu, Xuemei Ding, Girijesh Prasad, Magda Bucholc, David P. Finn, Stephen Todd, Paula L. McClean, KongFatt Wong-LinIEEE JBHI 2022 · DOI

Abstract

This study addresses the challenge of "extreme missingness" in dementia diagnosis, where roughly half of the clinical data is missing across a majority of features. By evaluating various imputation and classification workflows, the research demonstrates that while sophisticated imputation methods are accurate, a "reduced-feature" classification approach—building models on-the-fly using only available data—provides the optimal balance of diagnostic accuracy and computational speed for real-world clinical applications.

The Challenge of Extreme Missingness

In clinical practice, particularly within dementia care, datasets are rarely complete. Factors such as patient attrition, lengthy diagnostic wait times, and the inability of patients to complete taxing cognitive assessments lead to significant data gaps. This "extreme missingness"—often exceeding 50% across more than half of the features—poses a major hurdle for standard machine learning models, which typically require complete feature sets for prediction.

Actual clinical data illustrating extreme missingness patterns.

When a patient presents with incomplete test results, a Clinical Decision Support System (CDSS) must decide whether to "guess" missing values through imputation or adapt the model to the available data. This paper evaluates strategies for handling such missingness during both the training and real-time diagnostic phases.

Methodology and Datasets

The researchers compared multiple workflows combining imputation techniques and classification strategies. Imputation methods ranged from simple baselines (Mean, Median) to advanced iterative approaches like Predictive Mean Matching (PMM), Bayesian PCA, and missForest (using Random Forests). Classification was performed using Random Forests (RF), Support Vector Machines (SVM), and Naive Bayes (NB).

A key strategy tested was the "reduced-feature" approach, which avoids imputation for test patients by building a classification model using only the features present for that specific individual.

Simulated data used to test imputation and classification workflows.

Two primary datasets were utilized:

  • WHSCT (Altnagelvin Area Hospital): A real-world clinical dataset used to identify natural patterns of missing data.
  • ADNI (Alzheimer’s Disease Neuroimaging Initiative): A high-quality open dataset. A complete subset of ADNI served as the "ground truth," onto which the missingness patterns observed in WHSCT were synthetically applied for rigorous testing.

Key Insights and Practical Strategies

The study revealed several critical findings for clinical implementation:

  • Resilience of Accuracy: Surprisingly, diagnostic accuracy remained high (AUC > 0.85) even with 50% data missing. However, computational costs varied significantly.
  • Effectiveness of Reduced-Feature Models: Tailoring models to available data proved more effective than imputation during the test phase, as it prevents the propagation of errors inherent in "guessing" missing values.
  • Informative Partner Evaluations: Feature selection analysis indicated that assessments completed by caregivers (study partners) were often more predictive of disease severity than patient self-evaluations.
  • Naive Bayes as a Robust Baseline: Naive Bayes performed exceptionally well without imputation. Its independent treatment of features allows it to skip missing values, making it fast and robust for messy clinical data.

Results and Performance Comparison

The evaluation, conducted via Leave-One-Out Cross-Validation (LOOCV), highlighted two standout workflows:

  • Top Accuracy: Workflow L, combining Random Forest imputation on training data with a reduced-feature SVM classifier for testing, achieved the highest accuracy with an AUC of 0.896.
  • Best Practical Balance: Workflow B, utilizing Mean-by-class imputation and a reduced-feature Random Forest, offered a high AUC (0.878) with significantly lower computational overhead than iterative methods.

While Predictive Mean Matching (PMM50) was the most accurate at filling missing values ($R^2 = 0.86$), its high computational cost makes it less suitable for real-time clinical use compared to simpler methods that yield comparable diagnostic outcomes.

Conclusion and Future Directions

For real-world clinical settings, the "reduced-feature" approach combined with efficient imputation or robust classifiers like Naive Bayes offers the best trade-off between accuracy and speed. Future research should explore whether these strategies remain effective when incorporating high-dimensional data, such as genomics or neuroimaging (MRI/PET), and address the engineering challenges of storing or training reduced-feature models on-the-fly.

Summary updated

BibTeX
@article{mccombe2022practical,
  author = {Niamh McCombe and Shuo Liu and Xuemei Ding and Girijesh Prasad and Magda Bucholc and David P. Finn and Stephen Todd and Paula L. McClean and KongFatt Wong-Lin},
  title = {{Practical Strategies for Extreme Missing Data Imputation in Dementia Diagnosis}},
  journal = {IEEE J. Biomed. Health Informatics},
  year = {2022},
  doi = {10.1109/jbhi.2021.3098511},
}
Figure