Predicting Ischemic Stroke Outcome Using Deep Learning Approaches

Gang Fang, Zhennan Huang, Zhongrui WangFrontiers in Genetics 2022 · DOI

TL;DR

This paper investigates whether deep learning (DL) approaches can outperform traditional machine learning (ML) methods in predicting the 6-month functional outcome of ischemic stroke patients. Using the International Stroke Trial (IST) dataset, the study found that DL models did not significantly outperform ML models, suggesting that current DL methods may need further development for structured clinical data.

What Problem Does This Paper Address?

Predicting the functional outcome of ischemic stroke (IS) patients is crucial for setting realistic recovery goals, guiding rehabilitation plans, and managing patient and family expectations. This prediction helps physicians and caregivers make informed decisions about long-term care. While various clinical and radiological factors are used, accurately predicting these outcomes remains a challenge. The paper aims to explore the potential of modern deep learning techniques, which have shown great success in other domains like image and natural language processing, for this specific clinical prediction task and compare their performance against established machine learning methods.

Methods

The study employed three popular deep learning architectures: Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and Residual Neural Networks (ResNet). To adapt these models for structured, tabular clinical data, a one-dimensional convolutional layer was added as the initial layer for each. The researchers also utilized several classic machine learning algorithms, including Support Vector Machine (SVM), Random Forest (RF), and a more recent ensemble method called Deep Forest (DF). Feature selection was performed using Shapiro-Wilk analysis to identify and potentially remove less important features, leading to two sets of features: an initial set of 50 and a refined set of 44. The performance of all models was evaluated using metrics such as accuracy, precision, recall, and F1-score.

Datasets

The primary dataset used in this study is the International Stroke Trial (IST) dataset. This is a large, prospective, randomized controlled trial dataset collected between 1991 and 1996, containing 100% complete baseline data and over 99% complete follow-up data. The dataset includes baseline information and data collected 14 days post-randomization. For this study, the dataset was refined to include 16,403 patients with complete 6-month outcome data, categorized into three outcomes: dead, not recovered, and recovered.

Key Insights and Ideas

  • DL doesn't automatically win: Despite the hype around deep learning, this study found that for structured clinical data like the IST dataset, DL models did not significantly outperform traditional ML methods. This challenges the assumption that DL is universally superior.
  • Structured data is a different beast: DL models are often optimized for sequential or spatial data (like text or images). Their application to tabular, structured clinical data might require different architectural adaptations or preprocessing strategies than what was explored here.
  • Feature importance matters: Removing less important features (identified by Shapiro-Wilk analysis) did not negatively impact performance and, in some cases, seemed to allow DL models like ResNet to achieve better performance metrics, especially for predicting the "recovered" outcome.
  • Deep Forest shows promise: The Deep Forest (DF) algorithm, a novel ensemble method, demonstrated competitive performance, suggesting it's a strong contender for structured data prediction tasks.

Key Results

  • Comparable Accuracy: Both DL and ML models achieved similar prediction accuracies, generally around 82-83% for the test set using the initial 50 features.
  • No Clear DL Superiority: Across various metrics (precision, recall, F1-score), DL models did not consistently outperform ML models. In some instances, ML models, particularly Deep Forest, showed better performance in predicting the "recovered" outcome.
  • ResNet Improvement with More Training: While initial training of ResNet with 44 features yielded moderate results, training for 100 epochs led to a higher weighted average F1-score, suggesting that with appropriate training duration, ResNet could capture more nuanced patterns, though it still didn't surpass ML.
  • Overfitting Concerns: The study noted that DL models, particularly CNN and LSTM, began to overfit with extensive training (500 epochs), highlighting the need for careful hyperparameter tuning.

Limitations and Future Work

  • Data Age: The IST dataset is from the early 1990s. The authors acknowledge that some features might be outdated, and the limited availability of certain treatments (like thrombolysis) might affect prediction accuracy. Future work could involve using more recent datasets with contemporary treatment information.
  • DL Architecture Simplicity: The adapted DL architectures, while functional for structured data, were relatively simple (e.g., ResNet similar to ResNet18). The authors suggest exploring deeper DL frameworks and more powerful computing resources for future investigations.
  • Feature Engineering: While feature selection was performed, more advanced feature engineering or representation learning techniques could potentially enhance model performance.
  • Generalizability: The study focused on a single dataset. Future research should aim to validate these findings on diverse datasets to assess the generalizability of the models and conclusions.

Summary updated

Abstract

Predicting functional outcomes after an Ischemic Stroke (IS) is highly valuable for patients and desirable for physicians. This facilitates physicians to set reasonable goals for patients and cooperate with patients and relatives effectively, and furthermore to reach common after-stroke care decisions for recovery and make exercise plans to facilitate rehabilitation. The objective of this research is to apply three current Deep Learning (DL) approaches for 6-month IS outcome predictions, using the openly accessible International Stroke Trial (IST) dataset. Furthermore, another objective of this research is to compare these DL approaches with machine learning (ML) for performing in clinical prediction. After comparing various ML methods (Deep Forest, Random Forest, Support Vector Machine, etc.) with current DL frameworks (CNN, LSTM, Resnet), the results show that DL doesn't outperform ML significantly. DL methods and reporting used for analyzing structured medical data should be developed and improved.
BibTeX
@inproceedings{fang2022predicting,
  author = {Gang Fang and Zhennan Huang and Zhongrui Wang},
  title = {{Predicting Ischemic Stroke Outcome Using Deep Learning Approaches}},
  booktitle = {Frontiers in Genetics},
  year = {2022},
  doi = {10.3389/fgene.2021.827522},
}
Figure