Abstract
Accurate myocardium segmentation across all phases in one cardiac cycle in cine cardiac magnetic resonance (CMR) scans is crucial for comprehensively cardiac function analysis. Despite advancements in deep learning (DL) for automatic cine CMR segmentation, generalizability on unseen data remains a significant challenge. Recently, the segment-anything-model (SAM) has been invented as a segmentation foundation model, known for its accurate segmentation and more importantly, zero-shot generalization. SAM was trained on two-dimensional (2D) natural images; to adapt it for comprehensive cine CMR segmentation, we propose cineCMR-SAM which incorporates both temporal and spatial information through a modified model architecture. Compared to other state-of-the-art (SOTA) methods, our model achieved superior data-specific model segmentation accuracy on the STACOM2011 when fine-tuned on this dataset and demonstrated superior zero-shot generalization on two other large public datasets (ACDC and M&Ms) unseen during fine-tuning. Additionally, we introduced a text prompt feature in cineCMR-SAM to specify the view type of input slices (short-axis or long-axis), enhancing performance across all view types.
Introduction
Automating the segmentation of the myocardium in cine Cardiac Magnetic Resonance (CMR) is a cornerstone of modern cardiac disease diagnosis. However, current deep learning models face a persistent "generalizability gap," often failing when applied to images from different medical centers or scanner vendors (e.g., Siemens vs. GE). Furthermore, cine CMR is inherently 2D+Time (2D+T); models must maintain temporal consistency across the cardiac cycle to be clinically useful. Finally, while most tools focus on Short-Axis (SAX) views, Long-Axis (LAX) views are equally critical for metrics like longitudinal strain but are frequently neglected in automated pipelines.
Methodology: The cineCMR-SAM Architecture
The proposed cineCMR-SAM builds upon the Vision Transformer (ViT) backbone of Meta’s Segment Anything Model (SAM). To bridge the gap between static 2D natural images and dynamic 2D+T medical sequences, the authors introduced several key architectural innovations.
Temporal-Spatial Adaptation
Inspired by video processing architectures, the model utilizes modified ViT blocks that apply "Time-Space Self-Attention." This sequentially processes temporal dynamics across the cardiac cycle followed by spatial features within each slice. This allows the model to "understand" heart motion over time.
U-Net and Text Prompt Integration
To capture fine-grained anatomical details, refined SAM blocks are embedded within a 2D U-Net framework. Multi-Head Cross-Attention (MHCA) is used in the upsampling path to filter noise. Additionally, the model leverages SAM’s promptable nature by introducing a text encoder. By providing simple prompts like "SAX" or "LAX," the model adjusts its internal logic based on the specific anatomical view.
Parameter-Efficient Fine-Tuning
To preserve the extensive knowledge SAM gained from its original 1-billion-image training set, the authors employed a parameter-efficient strategy: freezing the original spatial weights while training only the new temporal modules, adapters, and U-Net layers.
Experimental Setup
The model was trained and fine-tuned on the STACOM 2011 dataset (100 patients, multi-center, multi-vendor). To test its robustness, zero-shot evaluations were performed on two external datasets:
- ACDC: 100 cases from a single center.
- M&Ms (Multi-Centre, Multi-Vendor & Multi-Disease): 136 cases from 5 centers and 4 vendors, serving as the ultimate test for cross-vendor generalizability.
Results and Discussion
cineCMR-SAM demonstrated superior performance and robustness compared to specialized medical imaging models like nnU-Net and U-transformer.
Key Findings
- Superior Accuracy: On the STACOM dataset, the model achieved a Dice score of 0.890. More importantly, it maintained high performance in zero-shot scenarios, achieving Dice scores of 0.850 on ACDC and 0.835 on M&Ms.
- Prompt Effectiveness: The inclusion of text prompts improved the Dice score for LAX views from 0.837 to 0.845 and significantly reduced boundary errors (Hausdorff Distance).
- Vendor Agnostic Performance: Performance remained consistent across GE, Philips, Siemens, and Canon scanners, proving the model's resilience to variations in image acquisition.
Conclusion
This study highlights the power of adapting foundation models for specialized medical tasks. By incorporating temporal-spatial attention and text-based context, cineCMR-SAM overcomes the traditional limitations of medical DL models regarding generalizability and view diversity. Future work will focus on automating the prompting process and incorporating anatomical priors to ensure segmentations remain physiologically consistent.
Summary updated