Predicting conversion from clinically isolated syndrome to multiple sclerosis–An imaging-based machine learning approach

Haike Zhang, Esther Alberts, Viola Pongratz, Mark Mühlau, Claus Zimmer, Benedikt Wiestler, Paul EichingerNeuroImage 2018 · DOI

TL;DR

This study leverages machine learning on baseline MRI scans to predict conversion from clinically isolated syndrome (CIS) to multiple sclerosis (MS) with 84.5% accuracy. By focusing on lesion shape features, the model outperforms traditional diagnostic criteria, offering a promising tool for early intervention in patients at risk of developing MS.

What Problem Does This Paper Address?

Clinically isolated syndrome (CIS) is often the first manifestation of multiple sclerosis (MS), but not all CIS patients go on to develop MS. Early identification of those who will convert is crucial because timely treatment can significantly alter disease progression and improve long-term outcomes. Current diagnostic methods, like the McDonald criteria, rely on observing "dissemination in space" and "dissemination in time" through MRI and clinical relapses. However, these criteria can be slow to confirm, and there's a need for more precise, earlier prediction tools. This paper aims to address this by exploring whether detailed analysis of brain lesions visible on initial MRI scans can predict future conversion to MS.

Methods

The researchers employed an oblique random forest classifier, a machine learning technique that builds multiple decision trees to make predictions. What makes it "oblique" is that at each decision point (node) in the tree, it uses a linear combination of features to make a split, rather than just a single feature. This can lead to more robust and accurate predictions, especially when features are correlated.

They extracted various features from brain lesions identified in baseline MRI scans. These features fell into two main categories:

  • Shape Features: These describe the geometric properties of the lesions, such as surface area, sphericity (how close to a sphere the lesion is), and surface-volume-ratio.
  • Intensity Features: These relate to the brightness of the lesions in the MRI images, including measures like skewness, kurtosis, and entropy of intensity histograms.

The study compared two approaches for lesion segmentation:

  1. Computer-assisted manual segmentation: A neuroradiologist manually outlined lesions, with some tool assistance.
  2. Automated segmentation: Using the Lesion Segmentation Toolbox (LST) for SPM, lesions were identified automatically.

The model was trained and validated using three-fold cross-validation to ensure reliable performance estimates. They also compared their machine learning approach against the traditional 2010 McDonald criteria (specifically, the "dissemination in space" aspect) as a benchmark.

Datasets

The study utilized a dataset of 84 patients diagnosed with CIS from a single-center prospective observational cohort. These patients were followed up for at least three years. Baseline MRI scans (3D FLAIR and 3D T1 sequences) were acquired between 2009 and 2013. The conversion to MS was determined based on the 2010 McDonald criteria. No new benchmark datasets were introduced; the study used its own collected patient data for analysis.

Key Insights and Ideas

  • Shape Trumps Intensity: The most significant finding is that lesion shape features are far more predictive of MS conversion than lesion intensity features. This suggests that the geometric characteristics of early inflammatory lesions carry crucial information about their future behavior.
  • Automated vs. Manual Segmentation: The performance of the shape-based classifier was comparable whether using computer-assisted manual segmentation or fully automated segmentation. This is a critical insight, as it implies that automated methods, which are much faster and scalable, can be reliably used for this predictive task.
  • Lesion Size Matters, But Not Count: While lesion volume (and related shape metrics) was important, the number of lesions did not emerge as a key predictor. This is counterintuitive, as one might expect more lesions to indicate a higher risk.
  • Beyond Visual Inspection: The study demonstrates the power of computational analysis to extract subtle, predictive information from MRI scans that might be missed or not fully utilized by human visual inspection alone.

Key Results

  • High Overall Accuracy: The shape-based random forest classifier achieved an accuracy of 84.5% in predicting conversion from CIS to MS.
  • Superior to McDonald Criteria: This accuracy significantly outperformed the prediction based on the dissemination in space criteria of the 2010 McDonald criteria, which achieved 75% accuracy.
  • Sensitivity and Specificity: The shape-based model achieved a sensitivity of 94% and a specificity of 50%. While the sensitivity is high (correctly identifying most converters), the specificity is moderate, meaning it also flagged some non-converters as converters.
  • Conversion Rate: In the study cohort, 79% of CIS patients converted to MS within three years.
  • Key Predictive Features: The most important shape features identified were mean lesion volume, minimum sphericity, and minimum surface-volume-ratio. Lesions in converters tended to be larger and less spherical.

Limitations and Future Work

  • Cohort Size: The study included 84 patients, which is a moderate size. The authors acknowledge that a larger, independent cohort would be ideal for external validation to confirm these findings and mitigate the risk of overfitting.
  • Feature Set: The analysis focused on pre-defined shape and intensity features. The potential contribution of other image features (e.g., texture) or more advanced deep learning approaches remains unexplored.
  • Transferability to 2D: The study used 3D MRI sequences. The authors note that extracting reliable shape features from small lesions might be challenging with 2D images, potentially limiting the direct transferability of this approach to older or different imaging protocols.
  • Future Directions: The authors suggest exploring the integration of clinical and paraclinical data (like age, gender, EDSS, CSF analysis) with imaging features to further refine prediction accuracy. They also propose moving beyond just predicting conversion to predicting different disease courses and identifying patients who would benefit most from early treatment.

Summary updated

Abstract

Magnetic resonance imaging (MRI) scans play a pivotal role in the evaluation of patients presenting with a clinically isolated syndrome (CIS), as these may depict brain lesions suggestive of an inflammatory cause. We hypothesized that it is possible to predict the conversion from CIS to multiple sclerosis (MS) based on the baseline MRI scan by studying image features of these lesions. We analyzed 84 patients diagnosed with CIS from a prospective observational single center cohort. The patients were followed up for at least three years. Conversion to MS was defined according to the 2010 McDonald criteria. Brain lesions were segmented based on 3D FLAIR and 3D T1 images. We generated brain lesion masks by a computer assisted manual segmentation. We also generated a set of automated segmentations using the Lesion Segmentation Toolbox for SPM to assess the influence of different segmentation methods. Shape and brightness features were automatically calculated from the segmented masks and used as input data to train an oblique random forest classifier. Prediction accuracies of the resulting model were validated through a three-fold cross-validation. Conversion from CIS to MS occurred in 66 of 84 patients (79%). The conversion or non-conversion was predicted correctly in 71 patients based on shape features derived from the computer assisted manual segmentation masks (84.5% accuracy). This predictor was more accurate than predicting conversion using dissemination in space at baseline according to the 2010 McDonald criteria (75% accuracy). While shape features strongly contributed to the accuracy of the predictor, including intensity features did not further improve performance. As patients who convert to definite MS benefit from early treatment, an early classification model is highly desirable. Our study shows that shape parameters of lesions can contribute to predicting the future course of CIS patients more accurately.
BibTeX
@inproceedings{zhang2018predicting,
  author = {Haike Zhang and Esther Alberts and Viola Pongratz and Mark Mühlau and Claus Zimmer and Benedikt Wiestler and Paul Eichinger},
  title = {{Predicting conversion from clinically isolated syndrome to multiple sclerosis–An imaging-based machine learning approach}},
  booktitle = {NeuroImage},
  year = {2018},
  doi = {10.1016/j.nicl.2018.11.003},
}
Figure