Multi-grained contrastive representation learning for label-efficient lesion segmentation and onset time classification of acute ischemic stroke

Jiarui Sun, Yuhao Liu, Yan Xi, Gouenou Coatrieux, Jean-Louis Coatrieux, Xu Ji, Liang Jiang, Yang ChenMedical Image Analysis 2024 · DOI

TL;DR

This paper introduces a novel "Multi-grained Contrastive Learning" (MGCL) framework to improve the segmentation of ischemic stroke lesions and predict the time since stroke onset, even with limited labeled data. By leveraging vast amounts of unlabeled MRI data, MGCL learns robust representations that significantly boost performance on these critical diagnostic tasks, ultimately aiming to improve stroke evaluation and treatment decisions.

What Problem Does This Paper Address?

Acute ischemic stroke (AIS) is a leading cause of death, and timely treatment, particularly with tissue plasminogen activator (tPA), is crucial. A key challenge in treating unwitnessed AIS patients is accurately determining the time since stroke onset (TSS), as tPA is most effective within a narrow time window (typically 4.5 hours). Current methods often rely on analyzing mismatch patterns between different MRI sequences (like DWI and FLAIR), which can be time-consuming and prone to inter-observer variability. While deep learning shows promise, it often struggles with generalization due to the scarcity of large, labeled multi-modal MRI datasets for AIS. This paper addresses this by proposing a method that effectively utilizes abundant unlabeled MRI data to learn powerful representations, thereby improving both lesion segmentation and TSS classification with limited labeled data.

Methods

The core of this work is the Multi-grained Contrastive Learning (MGCL) framework, which operates in two cascade stages:

  • Stage 1: Multi-grained Contrastive Learning: This stage focuses on learning "prior representations" from massive unlabeled MRI data. It employs two task-specific contrastive learning strategies:
  • Coarse-grained Version: This strategy uses slice-level information to learn global prior representations. It helps the model understand the overall location of ischemic lesions and the surrounding healthy tissue, crucial for AIS analysis.
  • Fine-grained Version: This strategy uses patch-level information to learn local prior representations. It enhances the model's ability to perceive the semantic relationships between the ischemic lesion and other healthy regions, capturing finer details.
  • Stage 2: Multi-task Learning: The prior representations learned in Stage 1 are transferred to a multi-task learning architecture. This architecture simultaneously performs:
  • Ischemic Lesion Segmentation (Task 1): Identifying the precise location and extent of the stroke lesion.
  • TSS Classification (Task 2): Predicting whether the stroke onset was within or beyond the critical 4.5-hour window.
  • Multi-modal Region-related Feature Fusion (MRFF) Module: This novel module is integrated into the multi-task framework. Inspired by Transformers, it captures feature correlations between corresponding regions of different MRI sequences (DWI, FLAIR, PWI). This allows it to explicitly incorporate clinical knowledge of mismatch patterns (like DWI-FLAIR or DWI-PWI mismatches) into the TSS decision-making process, improving accuracy.

Datasets

The authors constructed a standard multi-center multi-modal MRI acute ischemic stroke dataset (MMIS). This dataset includes:

  • Labeled Part: 327 paired multi-modal MRI images (DWI, FLAIR, PWI) with patient-level TSS classification labels (TSS < 4.5hrs or TSS ≥ 4.5hrs) and pixel-level ischemic lesion annotations.
  • Unlabeled Part: A large collection of unpaired MRI images, crucial for the self-supervised learning approach.

Key Insights and Ideas

  • Unlabeled Data is Gold: The paper strongly advocates for the underutilized potential of unlabeled medical imaging data. By effectively leveraging it through contrastive learning, significant performance gains can be achieved, especially in data-scarce domains like AIS.
  • Hierarchical Representation Learning: The "multi-grained" approach, combining coarse (global) and fine (local) representations, is key. This allows the model to grasp both the broad context of a stroke and the specific details of the lesion, leading to more robust feature learning.
  • Clinical Knowledge Integration: The MRFF module's ability to explicitly encode clinical mismatch patterns (like DWI-FLAIR) into the deep learning framework is a significant step towards bridging the gap between AI models and clinical diagnostic reasoning.
  • Label Efficiency: The MGCL framework demonstrates remarkable performance even with very limited labeled data, making it highly practical for real-world clinical scenarios where obtaining extensive annotations is challenging.

Key Results

  • Segmentation Performance: MGCL achieved state-of-the-art performance in ischemic lesion segmentation, outperforming baseline methods and other self-supervised/semi-supervised approaches, especially when training data was limited. For instance, with only 10% training data, MGCL achieved performance comparable to random initialization with 50% data on some datasets.
  • TSS Classification Performance: The proposed method achieved superior TSS classification results, with an accuracy of 0.844, sensitivity of 0.824, specificity of 0.867, and AUROC of 0.861, significantly outperforming existing methods, including the radiologist-derived DWI-FLAIR mismatch model.
  • Ablation Studies: Experiments confirmed the effectiveness of both the coarse-grained and fine-grained contrastive learning strategies, as well as the MRFF module, highlighting their individual contributions to the overall performance.

Limitations and Future Work

  • Limitations: The authors acknowledge that while their method significantly reduces dependence on labeled data, it still requires some labeled data for fine-tuning. The performance gains are more pronounced with less training data, suggesting that further optimization might be possible.
  • Future Work: The authors suggest exploring the integration of self-supervised and semi-supervised learning paradigms more deeply, as recent work indicates this combination can further boost the utilization of unlabeled data. They also plan to continue developing their approach for future research.

Summary updated

BibTeX
@article{sun2024multi,
  author = {Jiarui Sun and Yuhao Liu and Yan Xi and Gouenou Coatrieux and Jean-Louis Coatrieux and Xu Ji and Liang Jiang and Yang Chen},
  title = {{Multi-grained contrastive representation learning for label-efficient lesion segmentation and onset time classification of acute ischemic stroke}},
  journal = {Medical Image Analysis},
  year = {2024},
  doi = {10.1016/j.media.2024.103250},
}
Figure