New lesion segmentation for multiple sclerosis brain images with imaging and lesion-aware augmentation

Berke Doga Basaran, Paul M. Matthews, Wenjia BaiFrontiers in Neuroscience 2022 · DOI

TL;DR

Researchers have developed a high-performance deep learning pipeline for detecting newly formed Multiple Sclerosis (MS) lesions in longitudinal brain MRI scans. By building on the self-configuring nnU-Net framework and introducing specialized "lesion-aware" data augmentation techniques, the method significantly reduces false positives and outperforms both state-of-the-art architectures and human-expert-level benchmarks.

What Problem Does This Paper Address?

Multiple Sclerosis (MS) is a chronic inflammatory disease where the immune system attacks the central nervous system, leaving "lesions" visible on MRI. For clinicians, the appearance of new lesions between a baseline scan and a follow-up scan is a critical indicator of disease progression and treatment failure.

However, automating this detection is notoriously difficult for three reasons:

  1. Class Imbalance: New lesions are tiny, often occupying less than 0.01% of the total 3D brain volume. AI models struggle to find this "needle in a haystack."
  2. Imaging Variability: Scans from different hospitals use different magnets (1.5T vs 3T) and settings, leading to inconsistent image quality.
  3. Registration Errors: If the baseline and follow-up scans aren't perfectly aligned, the AI might mistake a slight shift in an old lesion for the appearance of a new one.

Methods

The authors proposed a pipeline centered on a 3D nnU-Net, a convolutional neural network (CNN) that automatically adapts its architecture to the dataset.

The core innovations lie in the preprocessing and augmentation stages:

  • Rigid and Non-Rigid Registration: The baseline scan is elastically "warped" to match the follow-up scan, helping the model ignore existing brain structures and focus solely on changes.
  • Axial Subsampling Augmentation: This simulates the blurring artifacts common in clinical MRI acquisition, making the model more robust to lower-quality scans.
  • CarveMix: This is a "lesion-aware" augmentation technique. The algorithm "carves" a lesion from one patient's scan and "mixes" it into another patient's healthy brain tissue during training. This artificially increases the number of training examples for rare lesion shapes and locations, directly tackling the class imbalance problem.

Datasets

The model was trained and evaluated using the MICCAI 2021 MS New Lesion Segmentation Challenge (MSSEG-2) dataset.

  • Composition: 100 MS patients scanned at two time points (1–3 years apart).
  • Diversity: Images were acquired from 15 different scanners (GE, Philips, Siemens) at both 1.5T and 3T strengths.
  • Ground Truth: Annotations were provided by four independent neuroradiologists, with a consensus mask used for final evaluation.

Key Insights and Ideas

  • Domain-Specific Augmentation > Model Complexity: While many researchers are moving toward complex Transformer-based architectures (like ViT), this paper shows that a standard CNN (nnU-Net) can win if it is fed better, more diverse data through techniques like CarveMix.
  • The "Growing Lesion" Trap: One of the most interesting findings was that models often fail by labeling the edges of growing lesions as new lesions. The authors used non-rigid registration to mitigate this, though it remains a challenge.
  • Robustness to "No-New-Lesion" Cases: Many AI models are "over-eager" and find lesions where none exist. This pipeline achieved a remarkably low false-positive rate, which is essential for clinical trust.

Key Results

The proposed method was ranked against 29 other participating teams in the MSSEG-2 challenge and several state-of-the-art architectures (e.g., TransBTS, UNETR).

  • Detection Performance: Achieved a Dice score of 0.510 and an $F_1$ score of 0.552 on cases with new lesions.
  • False Positive Control: In patients with no new lesions, the model was highly accurate, with an average false positive lesion number ($n_{FP}$) of only 0.036.
  • SOTA Comparison: The pipeline outperformed popular Transformer-based models, proving that specialized preprocessing and augmentation are currently more impactful in medical imaging than switching to "trendier" architectures.

Limitations and Future Work

  • Distinguishing Growth from New Formation: The model still occasionally struggles to differentiate between a brand-new lesion and the expansion of a pre-existing one.
  • Tiny Lesions: Very small or faint hyperintensities are still frequently missed by the automated system.
  • Anatomical Scope: The current study focuses exclusively on the brain. However, MS also affects the spinal cord. The authors suggest that future iterations should incorporate spinal cord MRI to provide a more holistic view of the disease.
  • Post-Processing: The authors noted that adding a dedicated post-processing step to "double-check" detected lesions against baseline locations could further reduce false positives.

Summary updated

Abstract

Multiple sclerosis (MS) is an inflammatory and demyelinating neurological disease of the central nervous system. Image-based biomarkers, such as lesions defined on magnetic resonance imaging (MRI), play an important role in MS diagnosis and patient monitoring. The detection of newly formed lesions provides crucial information for assessing disease progression and treatment outcome. Here, we propose a deep learning-based pipeline for new MS lesion detection and segmentation, which is built upon the nnU-Net framework. In addition to conventional data augmentation, we employ imaging and lesion-aware data augmentation methods, axial subsampling and CarveMix, to generate diverse samples and improve segmentation performance. The proposed pipeline is evaluated on the MICCAI 2021 MS new lesion segmentation challenge (MSSEG-2) dataset. It achieves an average Dice score of 0.510 and <i>F</i> <sub>1</sub> score of 0.552 on cases with new lesions, and an average false positive lesion number <i>n</i> <sub><i>FP</i></sub> of 0.036 and false positive lesion volume <i>V</i> <sub><i>FP</i></sub> of 0.192 <i>mm</i> <sup>3</sup> on cases with no new lesions. Our method outperforms other participating methods in the challenge and several state-of-the-art network architectures.
BibTeX
@inproceedings{basaran2022new,
  author = {Berke Doga Basaran and Paul M. Matthews and Wenjia Bai},
  title = {{New lesion segmentation for multiple sclerosis brain images with imaging and lesion-aware augmentation}},
  booktitle = {Frontiers in Neuroscience},
  year = {2022},
  doi = {10.3389/fnins.2022.1007453},
}
Figure