BeSt-LeS: Benchmarking Stroke Lesion Segmentation using Deep Supervision

Prantik Deb, Lalith Bharadwaj Baru, Kamalaker Dadi, Bapi Raju Surampudi

Abstract

Accurate segmentation of stroke lesions from MRI scans is vital for clinical decision-making and patient prognosis. However, the field lacks standardized benchmarks for the recently released ATLAS v2.0 dataset. This paper introduces BeSt-LeS, a comprehensive evaluation of various 2D and 3D U-Net architectures. Our findings indicate that hybrid transformer-convolutional models achieve state-of-the-art performance in 2D segmentation, while residual connections offer superior stability for 3D volumetric data. This work establishes a reproducible baseline for future research in automated stroke lesion analysis.

1. Introduction

Brain strokes represent a global health crisis, serving as a leading cause of long-term disability and mortality. In clinical settings, the precise identification of lesion location and volume is paramount for risk stratification and the development of targeted treatment plans. While manual segmentation by expert neuroradiologists remains the gold standard, the process is labor-intensive, time-consuming, and prone to inter-observer variability.

Deep learning models, particularly those based on the U-Net architecture, have shown promise in automating this task. However, progress has been hindered by the reliance on older, smaller datasets like ATLAS v1.2. The release of ATLAS v2.0 (Anatomical Tracings of Lesions After Stroke) provides a significantly larger and more diverse dataset, yet it lacks a standardized performance baseline for common segmentation architectures. BeSt-LeS addresses this gap by benchmarking "off-the-shelf" models to provide a clear starting point for the research community.

2. Methodology

We evaluated several variants of the U-Net architecture, categorized by their dimensionality and structural components.

2.1 Model Architectures

  • 2D Modality: We compared pure convolutional models—including the Standard U-Net, Residual U-Net, and Attention U-Net—against hybrid models that integrate Transformer components, specifically the TransAttn U-Net and the U-Net Transformer.
  • 3D Modality: To capture volumetric context, we evaluated 3D versions of the Standard, Residual, and Attention U-Nets.

2.2 Loss Function and Training

Models were trained using a composite loss function combining Dice Loss (to optimize for spatial overlap) and Binary Cross-Entropy (to ensure pixel-wise accuracy). A 90% weight was assigned to the Dice component to address the inherent class imbalance between lesion and non-lesion pixels. Notably, we omitted data augmentation to ensure the benchmark reflects the raw performance of the architectures on the provided data.

3. Experimental Setup

3.1 Dataset

The study utilized the ATLAS v2.0 dataset, which contains 1,271 T1-weighted MRI samples across 44 cohorts. For this benchmark, we focused on the 655 samples with publicly available lesion masks. The data was partitioned into a 60/20/20 split for training, validation, and testing, respectively.

3.2 Pre-processing

All images underwent Z-score normalization. For the 2D models, slices were resized to 192x192 pixels. For 3D models, volumes were processed at a resolution of approximately 144x172x128 voxels.

4. Results and Analysis

4.1 Quantitative Performance

Performance was primarily assessed using the Dice Similarity Coefficient (DSC).

  • 2D Performance: The U-Net Transformer emerged as the top performer with a Dice score of 0.583, outperforming the standard U-Net baseline by over 15%. This suggests that self-attention mechanisms are highly effective at capturing the long-range structural relationships necessary for brain segmentation.
  • 3D Performance: The 3D Residual U-Net led the volumetric category with a Dice score of 0.504. While 3D models generally improved performance for standard and residual architectures (by 4-7% over their 2D counterparts) due to spatial context, the Attention U-Net saw a performance decline in 3D, likely due to increased parameter complexity without the benefit of data augmentation.

4.2 Qualitative Observations

A significant finding across all models was the "Small Lesion Struggle." While architectures performed reliably on large, contiguous lesions, they frequently failed to detect or accurately segment small, isolated lesions. However, statistical analysis using the Wilcoxon Signed Rank test for the Attention U-Net (p=0.54) indicated no significant difference between predicted and actual total stroke volumes, suggesting these models remain useful for estimating total lesion load.

5. Conclusion and Future Work

BeSt-LeS establishes a rigorous baseline for stroke lesion segmentation on ATLAS v2.0. Our results highlight the superiority of hybrid Transformer models in 2D and Residual models in 3D. Future work will focus on addressing the detection of small lesions and exploring weakly-supervised learning techniques to leverage the remaining 300+ unlabeled images in the ATLAS v2.0 dataset. Additionally, incorporating uncertainty estimation will be a critical step toward making these models clinically viable.

Summary updated

Abstract

Brain stroke has become a significant burden on global health and thus we need remedies and prevention strategies to overcome this challenge. For this, the immediate identification of stroke and risk stratification is the primary task for clinicians. To aid expert clinicians, automated segmentation models are crucial. In this work, we consider the publicly available dataset ATLAS v2.0 to benchmark various end-to-end supervised U-Net style models. Specifically, we have benchmarked models on both 2D and 3D brain images and evaluated them using standard metrics. We have achieved the highest Dice score of 0.583 on the 2D transformer-based model and 0.504 on the 3D residual U-Net respectively. We have conducted the Wilcoxon test for 3D models to correlate the relationship between predicted and actual stroke volume. For reproducibility, the code and model weights are made publicly available.
BibTeX
@misc{deb2023best,
  author = {Prantik Deb and Lalith Bharadwaj Baru and Kamalaker Dadi and Bapi Raju Surampudi},
  title = {{BeSt-LeS: Benchmarking Stroke Lesion Segmentation using Deep Supervision}},
  year = {2023},
  eprint = {2310.07060},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.07060},
}
Figure