Predicting Progression from Normal to MCI and from MCI to AD Using Clinical Variables in the National Alzheimer's Coordinating Center Uniform Data Set Version 3: Application of Machine Learning Models and a Probability Calculator

Yana Pang, Walter A. Kukull, Mary Sano, Roger L. Albin, Changyu Shen, Jiayu Zhou, Hiroko H. DodgeThe Journal of Prevention of Alzheimer s Disease 2023 · DOI

TL;DR

This paper presents a machine learning approach to predict the progression of Alzheimer's disease (AD) from normal cognition to Mild Cognitive Impairment (MCI), and from MCI to AD, using only clinical data. They've developed a user-friendly calculator based on these models, which can help identify individuals at high risk for progression, aiding in clinical trial recruitment and patient management.

What Problem Does This Paper Address?

Identifying individuals who are likely to progress from normal cognition to MCI, or from MCI to AD, is crucial for the success of clinical trials and for providing timely interventions. Current methods often rely on expensive and invasive biomarkers, which may not always correlate perfectly with clinical outcomes. Furthermore, the significant variability between individuals makes it challenging to predict progression solely based on biomarkers or comorbidities. This study aims to address this by developing accurate predictive models using readily available clinical data, making the process more accessible and efficient.

Methods

The researchers employed machine learning techniques to predict clinical progression. They explored various classifiers, including Support Vector Machines (SVM), Logistic Regression (LR), and Random Forests, and evaluated different feature selection methods like Information Gain, chi-squared test, Fisher Score, and ANOVA. They also incorporated embedded methods like LASSO and Decision Trees for joint feature selection. The models were trained and tested using a 5-fold cross-validation approach, repeated 10 times to ensure robustness. A key aspect of their work is the development of a user-friendly calculator that takes a small set of selected clinical variables and outputs a probability of conversion.

Datasets

The study utilized data from the National Alzheimer's Coordinating Center (NACC) Uniform Data Set Version 3 (UDS V3). This dataset comprises systematically collected data from participants across various Alzheimer's Disease Centers (ADCs) in the United States. The data includes demographics, medical history, neurological exams, cognitive ratings (like the Clinical Dementia Rating - CDR), and neuropsychological test scores. The UDS V3 is an updated version of the dataset used in their previous work.

Key Insights and Ideas

  • Clinical data is powerful: The study demonstrates that a carefully selected set of clinical variables, without relying on expensive biomarkers, can achieve high accuracy in predicting AD progression.
  • Clinician's judgment is key: The variable "COGSTAT" (clinician's judgment of symptoms) consistently emerged as a significant predictor across different transition models, highlighting the value of expert clinical assessment.
  • Short-term prediction is feasible: The models can predict progression within relatively short timeframes (2-3 years for MCI to AD), which is highly relevant for clinical trial durations.
  • A practical tool for researchers and clinicians: The developed probability calculator offers a tangible tool for pre-screening participants for trials or for stratifying patients for further assessment.

Key Results

The study achieved promising results in predicting transitions:

  • Normal Cognition to MCI (NC2MCI): The Area Under the Curve (AUC) for predicting this transition was around 70%.
  • MCI to AD: The AUC for predicting MCI to AD transition was higher, around 80%. Specifically, for the MCI to AD transition within 2 years (MCI2AD_2), the best models achieved an accuracy of 85% with high sensitivity and specificity.
  • Selected Variables: A set of 20 clinical variables was found to be optimal for prediction, with clinician's judgment (COGSTAT) being a consistently important predictor. Interestingly, for MCI to AD transition within 2 years, smoking status in the last 30 days was also identified as a predictor, with those who did not smoke being more likely to transition.

Limitations and Future Work

The authors acknowledge several limitations:

  • Generalizability: The NACC cohort is not an epidemiological sample and is skewed towards higher income and education. Therefore, the generalizability of the results to the broader elderly population is limited. Certain racial and ethnic groups were also underrepresented.
  • Linear Models: The study primarily used linear models, which might not capture complex non-linear relationships between predictor variables and outcomes.
  • Future Directions: The authors suggest that their calculator can be used for study enrichment and pre-screening for biomarker assessments. They also mention the potential for developing algorithms that mimic clinical judgment using information not captured in standard formats, such as speech characteristics. Further research could explore incorporating Electronic Health Records (EHR) for a more comprehensive longitudinal medical history.

Summary updated

Abstract

Clinical trials are increasingly focused on pre-manifest and early Alzheimer's disease (AD). Accurately predicting clinical progressions from normal to MCI or from MCI to dementia/AD versus non-progression is challenging. Accurate identification of symptomatic progressors is important to avoid unnecessary treatment and improve trial efficiency. Due to large inter-individual variability, biomarker positivity and comorbidity information are often insufficient to identify those destined to have symptomatic progressions. Using only clinical variables, we aimed to predict clinical progressions, estimating probabilities of progressions with a small set of variables selected by machine learning approaches. This work updates our previous work that was applied to the National Alzheimer's Coordinating Center (NACC) Uniform Data Set Version 2 (V2), by using the most recent version (V3) with additional analyses. We generated a user-friendly conversion probability calculator which can be used for effectively pre-screening trial participants.
BibTeX
@article{pang2023predicting,
  author = {Yana Pang and Walter A. Kukull and Mary Sano and Roger L. Albin and Changyu Shen and Jiayu Zhou and Hiroko H. Dodge},
  title = {{Predicting Progression from Normal to MCI and from MCI to AD Using Clinical Variables in the National Alzheimer's Coordinating Center Uniform Data Set Version 3: Application of Machine Learning Models and a Probability Calculator}},
  journal = {The Journal of Prevention of Alzheimer s Disease},
  year = {2023},
  doi = {10.14283/jpad.2023.10},
}
Figure