Data-driven biomarkers outperform theory-based biomarkers in predicting stroke motor outcomes

Emily Olafson, Christoph Sperber, Keith Jamison, Mark Bowren, Aaron D. Boes, Justin W. Andrushko, Michael R. Borich, Lara A. Boyd, Jessica M. Cassidy, Adriana Bastos Conforto, Steven C. Cramer, Adrienne N. Dula, Fatemeh Geranmayeh, Brenton Hordacre, Neda Jahanshad, Steven A. Kautz, Bethany Lo, Bradley J. MacIntosh, Fabrizio Piras, Andrew D. Robertson, Na Jin Seo, Surjo R. Soekadar, Sophia I. Thomopoulos, Daniela Vecchio, Timothy B. Weng, Lars T. Westlye, Carolee J. Winstein, George F. Wittenberg, Kristin A. Wong, Paul M. Thompson, Sook‐Lei Liew, Amy KuceyeskibioRxiv 2023 · DOI

TL;DR

This study demonstrates that data-driven approaches, particularly those analyzing structural disconnection in the brain, significantly outperform traditional theory-based biomarkers in predicting long-term motor outcomes after stroke. Combining these data-driven insights with demographic information further enhances prediction accuracy, paving the way for more personalized rehabilitation strategies.

What Problem Does This Paper Address?

Predicting how well stroke survivors will recover their motor function is crucial for tailoring rehabilitation and guiding clinical trials. Traditionally, this prediction has relied on "theory-based" biomarkers, which focus on damage to specific, pre-defined motor pathways like the corticospinal tract. However, these approaches may not fully capture the complex ways a stroke can impact motor control, especially considering the diverse lesion locations and the interconnectedness of brain networks. The authors aim to explore whether "data-driven" methods, which learn associations directly from the data without pre-existing assumptions about critical pathways, can provide more accurate predictions of chronic motor outcomes.

Methods

The researchers employed a rigorous cross-validation framework to train and test various predictive models. They compared several types of biomarkers:

  • Theory-based:
    • M1-CST-LL: Lesion load in the primary motor cortex's corticospinal tract.
  • SMATT-LL: Lesion load across multiple tracts in the sensorimotor system.
  • Data-driven:
    • LBM-LL: Lesion load of "lesion-behavior maps" (LBMs), which identify brain regions where damage is statistically associated with motor deficits.
  • sLNM-LL: Lesion load of "structural lesion-network maps" (sLNMs), which map white matter networks associated with peak LBM regions.
  • ChaCo scores: Measures of "Change in Connectivity" derived from the Network Modification (NeMo) tool, quantifying structural disconnection in specific brain regions.

They also explored ensemble models that combined predictions from multiple biomarkers, including demographic data (age, sex, time since stroke), to see if this integration improved accuracy.

Datasets

The study utilized data from the ENIGMA Stroke Recovery Working Group database. This dataset comprises structural MRI scans and motor outcome measures from 789 stroke patients. The data was collected from 22 different research studies.

Key Insights and Ideas

  • Beyond the Obvious: The findings suggest that damage outside the primary motor system (e.g., in frontal and parietal lobes involved in planning and attention) can be highly predictive of motor outcomes. This highlights the importance of considering the brain as a complex, interconnected network rather than isolated pathways.
  • Structural Disconnection is Key: Measures of how a lesion disrupts brain connectivity (ChaCo scores) proved to be the most powerful single predictor, outperforming simpler lesion load measures. This indicates that the functional impact of a lesion on network integrity is more informative than just the amount of tissue damage.
  • Data-Driven Dominance: The study strongly supports the superiority of data-driven approaches over traditional theory-based biomarkers for predicting chronic motor deficits. This suggests that machine learning can uncover subtle but important lesion-deficit relationships that might be missed by pre-defined anatomical targets.
  • The Power of Combination: While data-driven models excelled individually, combining predictions from multiple sources (different biomarkers and demographic data) yielded the best overall predictive performance. This emphasizes that a comprehensive view, integrating various types of information, is essential for accurate prediction.

Key Results

  • Data-driven biomarkers consistently outperformed theory-based biomarkers. The best data-driven model, using regional structural disconnection (ChaCo scores), achieved an R² of 0.210, significantly higher than the best theory-based models (M1-CST-LL: R² = 0.132; SMATT-LL: R² = 0.180).
  • Regional structural disconnection (ChaCo scores) was the top-performing individual biomarker.
  • Ensemble models significantly improved prediction accuracy. Combining demographic information with the best-performing ChaCo models resulted in the highest prediction accuracy, achieving an R² of 0.241.
  • Lesion-behavior maps (LBM-LL) showed surprisingly strong performance, often comparable to more complex data-driven models, suggesting their utility as a simpler, interpretable biomarker.

Limitations and Future Work

  • Lack of Baseline Motor Scores: The study could not directly compare the predictive power of lesion damage versus baseline behavioral information, which could share variance and influence outcomes. Future work should incorporate baseline scores to disentangle these effects.
  • Rehabilitation Variability: The study acknowledges that differences in rehabilitation levels among participants could impact chronic motor scores, introducing a source of unexplained variance. Future models could benefit from including rehabilitation data.
  • Generalizability of Paradoxical Weights: While some "paradoxical" feature weights (where damage to a region was associated with better outcomes) were observed, their stability and generalizability across different subsamples require further investigation.
  • Data Sharing Restrictions: Due to privacy concerns, individual subject data is not publicly available, which limits direct replication by other researchers. However, data sharing is possible for ENIGMA members under specific agreements.
  • Future Directions: The authors suggest that future work should explore combining multiple features and feature representations, as well as incorporating more recovery-relevant variables, to further enhance predictive accuracy. Investigating the precise neural correlates of motor deficits and optimizing model interpretation strategies are also highlighted as important future steps.

Summary updated

Abstract

Chronic motor impairments are a leading cause of disability after stroke. Previous studies have predicted motor outcomes based on the degree of damage to predefined structures in the motor system, such as the corticospinal tract. However, such theory-based approaches may not take full advantage of the information contained in clinical imaging data. The present study uses data-driven approaches to predict chronic motor outcomes after stroke and compares the accuracy of these predictions to previously-identified theory-based biomarkers. Using a cross-validation framework, regression models were trained using lesion masks and motor outcomes data from 789 stroke patients (293 female/496 male) from the ENIGMA Stroke Recovery Working Group (age 64.9±18.0 years; time since stroke 12.2±0.2 months; normalised motor score 0.7±0.5 (range [0,1]). The out-of-sample prediction accuracy of two theory-based biomarkers was assessed: lesion load of the corticospinal tract, and lesion load of multiple descending motor tracts. These theory-based prediction accuracies were compared to the prediction accuracy from three data-driven biomarkers: lesion load of lesion-behaviour maps, lesion load of structural networks associated with lesion-behaviour maps, and measures of regional structural disconnection. In general, data-driven biomarkers had better prediction accuracy - as measured by higher explained variance in chronic motor outcomes - than theory-based biomarkers. Data-driven models of regional structural disconnection performed the best of all models tested (R<sup>2</sup> = 0.210, p < 0.001), performing significantly better than predictions using the theory-based biomarkers of lesion load of the corticospinal tract (R<sup>2</sup> = 0.132, p< 0.001) and of multiple descending motor tracts (R<sup>2</sup> = 0.180, p < 0.001). They also performed slightly, but significantly, better than other data-driven biomarkers including lesion load of lesion-behaviour maps (R<sup>2</sup> =0.200, p < 0.001) and lesion load of structural networks associated with lesion-behaviour maps (R<sup>2</sup> =0.167, p < 0.001). Ensemble models - combining basic demographic variables like age, sex, and time since stroke - improved prediction accuracy for theory-based and data-driven biomarkers. Finally, combining both theory-based and data-driven biomarkers with demographic variables improved predictions, and the best ensemble model achieved R<sup>2</sup> = 0.241, p < 0.001. Overall, these results demonstrate that models that predict chronic motor outcomes using data-driven features, particularly when lesion data is represented in terms of structural disconnection, perform better than models that predict chronic motor outcomes using theory-based features from the motor system. However, combining both theory-based and data-driven models provides the best predictions.
BibTeX
@misc{olafson2023data,
  author = {Emily Olafson and Christoph Sperber and Keith Jamison and Mark Bowren and Aaron D. Boes and Justin W. Andrushko and Michael R. Borich and Lara A. Boyd and Jessica M. Cassidy and Adriana Bastos Conforto and Steven C. Cramer and Adrienne N. Dula and Fatemeh Geranmayeh and Brenton Hordacre and Neda Jahanshad and Steven A. Kautz and Bethany Lo and Bradley J. MacIntosh and Fabrizio Piras and Andrew D. Robertson and Na Jin Seo and Surjo R. Soekadar and Sophia I. Thomopoulos and Daniela Vecchio and Timothy B. Weng and Lars T. Westlye and Carolee J. Winstein and George F. Wittenberg and Kristin A. Wong and Paul M. Thompson and Sook‐Lei Liew and Amy Kuceyeski},
  title = {{Data-driven biomarkers outperform theory-based biomarkers in predicting stroke motor outcomes}},
  year = {2023},
  doi = {10.1101/2023.06.19.545638},
  howpublished = {bioRxiv},
}

Loading related content...

Figure