Predicting response to motor therapy in chronic stroke patients using Machine Learning

Ceren Tozlu, Dylan J. Edwards, Aaron D. Boes, Douglas Labar, K. Zoe Tsagaris, Joshua R. Silverstein, Heather Lane, Mert R. Sabuncu, Charles Liu, Amy KuceyeskibioRxiv 2018 · DOI

TL;DR

This study leverages machine learning to predict motor recovery in chronic stroke patients undergoing therapy. By analyzing demographic, clinical, and neurophysiological data, they found that Elastic-Net models, particularly those incorporating pre-intervention Fugl-Meyer Assessment and motor threshold differences, offered the most accurate predictions, outperforming models that included imaging data.

What Problem Does This Paper Address?

Predicting how well a stroke patient will recover motor function after therapy is crucial for tailoring treatment plans and managing expectations. However, for chronic stroke patients, where recovery gains are often incremental, this prediction is particularly challenging for clinicians. Existing methods often fall short at an individual patient level. This paper aims to improve these predictions by applying various machine learning techniques to a comprehensive set of patient data, seeking to identify the most influential factors for motor improvement.

Methods

The researchers employed five distinct machine learning methods: Elastic-Net (EN), Support Vector Machines (SVM), Artificial Neural Networks (ANN), Classification and Regression Trees (CART), and Random Forest (RF). They used a k-fold cross-validation approach (100 outer loops, 10 inner loops) to train and test their models. Model performance was evaluated using R-squared (explained variance) and Root Mean Squared Error (RMSE) for regression tasks, and Area Under the ROC Curve (AUC) for classification tasks. They also investigated variable importance within the models to understand which factors contributed most to the predictions.

Datasets

The study utilized data from 102 chronic stroke patients who had suffered their first ischemic stroke 3-12 months prior. The dataset included:

  • Demographic data: Age, sex, handedness.
  • Clinical data: Time since stroke, stroke lateralization (left vs. right hemisphere), pre-intervention Fugl-Meyer Assessment (FMA) scores.
  • Neurophysiological data: Difference in motor threshold (MT) between unaffected and affected hemispheres, and the presence or absence of motor evoked potential (MEP) in the affected hemisphere.
  • Imaging data: Structural MRI scans were used to derive measures of regional and pair-wise disconnectivity using the Network Modification (NeMo) Tool.

Key Insights and Ideas

  • Neurophysiological data reigns supreme: The study strongly suggests that neurophysiological measures, particularly the difference in motor threshold between hemispheres, are more powerful predictors of motor recovery than demographic or even imaging-based disconnectivity measures.
  • Elastic-Net's efficiency: The Elastic-Net model consistently outperformed other machine learning methods, likely due to its ability to handle sparsity and select a parsimonious set of strong predictors.
  • Imaging data's limited added value: While imaging-based disconnectivity measures were explored, they generally did not significantly improve prediction accuracy when added to clinical and neurophysiological data, suggesting that current imaging metrics might not capture the most critical aspects of recovery in this context.
  • Pre-intervention FMA is a cornerstone: Unsurprisingly, the baseline Fugl-Meyer Assessment score was a critical predictor across all models, highlighting the importance of initial motor impairment severity.

Key Results

  • Best overall performance: Elastic-Net (EN) achieved the highest R² values across all model types (clinical, clinical + regional disconnectivity, clinical + pair-wise disconnectivity), indicating it explained the most variance in post-intervention FMA. For the clinical model, EN achieved an R² of 0.910.
  • Predictive power of clinical variables: In the "clinical" model (demographic, clinical, and neurophysiological data), EN performed significantly better than other methods. Pre-intervention FMA and the difference in motor threshold were identified as the strongest predictors.
  • Classification accuracy: For classifying patients into "good" vs. "poor" recovery, EN, ANN, and CART showed satisfactory results, with AUCs generally above 0.6. ANN and CART showed improvement when pairwise disconnectivity data was included.
  • Post-hoc analysis: Adding the difference in motor threshold to a model based solely on pre-intervention FMA significantly improved prediction accuracy, whereas adding the presence/absence of MEP did not.

Limitations and Future Work

  • Limited imaging data: The study primarily used structural T1 scans. Future work could benefit from incorporating diffusion or functional MRI data, which have shown promise in identifying biomarkers for impairment and recovery.
  • Sample size: While relatively large for this type of study, the authors acknowledge that machine learning techniques like deep learning might require even larger datasets to achieve optimal accuracy.
  • Multi-modal data integration: Future studies in this cohort will focus on collecting multi-modal imaging data at baseline and post-intervention to enhance prediction accuracy and explore recovery mechanisms.
  • Validation: Thorough validation of these machine learning models using larger, multi-modal datasets is needed to confirm their clinical utility.

Summary updated

Abstract

Abstract Accurate predictions of motor improvement resulting from intensive therapy in chronic stroke patients is a difficult task for clinicians, but is key in prescribing appropriate therapeutic strategies. Statistical methods, including machine learning, are a highly promising avenue with which to improve prediction accuracy in clinical practice. The first main objective of this study was to use machine learning methods to predict a chronic stroke individual’s motor function improvement after 6 weeks of intervention using pre-intervention demographic, clinical, neurophysiological and imaging data. The second main objective was to identify which data elements were most important in predicting chronic stroke patients’ impairment after 6 weeks of intervention. Data from one hundred and two patients (Female: 31%, age 61±11 years) who suffered first ischemic stroke 3-12 months prior were included in this study. After enrollment, patients underwent 6 weeks of intensive motor and transcranial magnetic stimulation therapy. Age, gender, handedness, time since stroke, pre-intervention Fugl-Meyer Assessment, stroke lateralization, the difference in motor threshold between the unaffected and affected hemispheres, absence or presence of motor evoked potential in the affected hemisphere and various imaging metrics were used as predictors of post-intervention Fugl-Meyer Assessment. Five machine learning methods, including Elastic-Net, Support Vector Machines, Artificial Neural Networks, Classification and Regression Trees, and Random Forest, were used to predict post-intervention Fugl-Meyer Assessment based on either demographic, clinical and neurophysiological data alone or in combination with the imaging metrics. Cross-validated R-squared and root of mean squared error were used to assess the prediction accuracy and compare the performance of methods. Elastic-Net performed significantly better than the other methods for the model containing pre-intervention Fugl-Meyer Assessment, demographic, clinical and neurophysiological data as predictors of post-intervention Fugl-Meyer Assessment ( ). Pre-intervention Fugl-Meyer Assessment and difference in motor threshold between affected and unaffected hemispheres were commonly found as the strongest two predictors in the clinical model. The difference in motor threshold had greater importance than the absence or presence of motor evoked potential in the affected hemisphere. The various imaging metrics, including lesion overlap with the spinal cord, largely did not improve the model performance. The approach implemented here may enable clinicians to more accurately predict a chronic stroke patient’s individual response to intervention. The predictive models used in this study could assist clinicians in making treatment decisions and improve the accuracy of prognosis in chronic stroke patients.
BibTeX
@misc{tozlu2018predicting,
  author = {Ceren Tozlu and Dylan J. Edwards and Aaron D. Boes and Douglas Labar and K. Zoe Tsagaris and Joshua R. Silverstein and Heather Lane and Mert R. Sabuncu and Charles Liu and Amy Kuceyeski},
  title = {{Predicting response to motor therapy in chronic stroke patients using Machine Learning}},
  year = {2018},
  doi = {10.1101/457416},
  howpublished = {bioRxiv},
}
Figure