MyoPS-Net: Myocardial pathology segmentation with flexible combination of multi-sequence CMR images

Junyi Qiu, Lei Li, Sihan Wang, Ke Zhang, Yinyin Chen, Shan Yang, Xiahai Zhuang

TL;DR

MyoPS-Net is a new end-to-end deep learning framework designed to segment myocardial scars and edema from multiple types of cardiac MRI sequences. Its breakthrough lies in its "flexible combination" architecture, which allows it to maintain high accuracy even when certain imaging sequences (like contrast-enhanced scans) are missing—a common occurrence in real-world clinical settings.

What Problem Does This Paper Address?

Accurately identifying myocardial scars and edema is critical for diagnosing and treating myocardial infarction (heart attack). Doctors typically use several Cardiac Magnetic Resonance (CMR) sequences to see different things: LGE (Late Gadolinium Enhancement) is the gold standard for scars, while T2-weighted images highlight edema.

However, two major hurdles exist:

  1. Information Fusion: It is difficult to effectively combine data from five different CMR sequences (bSSFP, LGE, T2, T1 mapping, and T2* mapping) because they have different appearances and noise levels.
  2. Missing Data: In the real world, patients often lack a full set of scans. For instance, patients with kidney issues cannot receive the contrast agents required for LGE scans. Most existing AI models break down if even one expected input is missing.

Methods

The authors propose MyoPS-Net, which introduces three novel components to handle multi-sequence data:

  • Cross-Modal Feature Fusion (CMFF): Instead of just stacking images at the beginning, this module uses multiple encoders that "talk" to each other via pixel-wise max operations. This allows the network to extract the best features from whatever sequences are available.
  • Myocardium Prior and Consistency (MPC) Module: Since scars and edema must exist within the heart muscle (myocardium), this module first localizes the myocardium. It then uses a "consistency loss" to ensure that the pathology segmentation stays strictly within those anatomical boundaries.
  • Pathology Inclusiveness (PI) Loss: This is a clever "anatomical rule" baked into the math. In medicine, myocardial scars are almost always contained within a larger area of edema. The PI loss penalizes the model if it predicts a scar outside of an edematous region, forcing the AI to learn realistic spatial relationships.

Datasets

The model was evaluated on two primary sources:

  1. Private Dataset: 50 paired multi-sequence CMR images from Fudan University, featuring all five sequences (bSSFP, LGE, T2, T1 mapping, and T2* mapping).
  2. Public Dataset: The MICCAI 2020 MyoPS Challenge dataset, which is a standard benchmark in the field for myocardial pathology segmentation.

Key Insights and Ideas

  • Anatomy as a Constraint: The most powerful part of the paper is the shift from "pure" computer vision to "physically constrained" vision. By telling the model that "scars live inside edema" and "everything lives inside the muscle," the researchers significantly reduced false positives.
  • Clinical Flexibility: The authors didn't just build one model; they tested variants (MyoPS-Net-L for LGE-heavy workflows and MyoPS-Net-M for mapping-heavy workflows). They proved that a "unified" model trained on mixed data (MyoPS-Net-mix) can generalize across different hospital protocols.
  • Mapping as a Substitute: The results suggest that T1 and T2* mapping can provide valuable pathological information even without contrast agents, though they are currently less precise than LGE.

Key Results

  • State-of-the-Art Performance: MyoPS-Net achieved a Dice score (a measure of overlap where 1.0 is perfect) of 0.661 for scars and 0.742 for edema on the public challenge dataset, outperforming several established models like nnU-Net and EfficientSeg.
  • Ablation Success: Each module added value. Adding the CMFF module boosted the scar Dice score from 0.560 to 0.614, and adding the MPC and PI modules pushed it further to 0.656.
  • Robustness to Missing Modalities: Even when LGE (the most informative sequence) was removed, the model could still produce clinically relevant segmentations using mapping sequences, which is a major win for "contrast-free" imaging.

Limitations and Future Work

  • Image Registration: The paper assumes that all CMR sequences are already perfectly aligned (registered). In practice, heart motion and breathing make this very difficult. The authors acknowledge that a unified "registration + segmentation" framework is the next logical step.
  • Mapping Quality: Mapping CMR images are often blurry or low-resolution. The authors suggest using Cine CMR (moving images of the heart) in the future to help the model better understand the fine details of mapping sequences.
  • Small Sample Size: While 50 cases is decent for specialized medical imaging, larger multi-center datasets would be needed to prove the model works across different MRI scanner brands (e.g., Siemens vs. GE).

Summary updated

Abstract

Myocardial pathology segmentation (MyoPS) can be a prerequisite for the accurate diagnosis and treatment planning of myocardial infarction. However, achieving this segmentation is challenging, mainly due to the inadequate and indistinct information from an image. In this work, we develop an end-to-end deep neural network, referred to as MyoPS-Net, to flexibly combine five-sequence cardiac magnetic resonance (CMR) images for MyoPS. To extract precise and adequate information, we design an effective yet flexible architecture to extract and fuse cross-modal features. This architecture can tackle different numbers of CMR images and complex combinations of modalities, with output branches targeting specific pathologies. To impose anatomical knowledge on the segmentation results, we first propose a module to regularize myocardium consistency and localize the pathologies, and then introduce an inclusiveness loss to utilize relations between myocardial scars and edema. We evaluated the proposed MyoPS-Net on two datasets, i.e., a private one consisting of 50 paired multi-sequence CMR images and a public one from MICCAI2020 MyoPS Challenge. Experimental results showed that MyoPS-Net could achieve state-of-the-art performance in various scenarios. Note that in practical clinics, the subjects may not have full sequences, such as missing LGE CMR or mapping CMR scans. We therefore conducted extensive experiments to investigate the performance of the proposed method in dealing with such complex combinations of different CMR sequences. Results proved the superiority and generalizability of MyoPS-Net, and more importantly, indicated a practical clinical application.
BibTeX
@article{qiu2023myops,
  author = {Junyi Qiu and Lei Li and Sihan Wang and Ke Zhang and Yinyin Chen and Shan Yang and Xiahai Zhuang},
  title = {{MyoPS-Net: Myocardial pathology segmentation with flexible combination of multi-sequence CMR images}},
  journal = {Medical Image Analysis},
  year = {2023},
  doi = {10.1016/j.media.2022.102694},
}

Loading related content...

Figure