TL;DR
Deep learning models excel at cardiac MRI analysis but frequently degrade when images are blurred by patient breathing. The CMRxMotion challenge introduces a newly curated dataset with controlled respiratory motion artifacts to benchmark automated image quality assessment and robust myocardial segmentation. The findings show that while AI approaches human-level performance on clean scans, respiratory motion significantly compromises both segmentation accuracy and downstream clinical biomarkers.
What Problem Does This Paper Address?
Automated Cardiac Magnetic Resonance (CMR) image analysis relies heavily on high-quality, artifact-free images. In clinical settings, however, patients—especially pediatric patients or those suffering from heart failure—often struggle to follow strict breath-holding instructions. This results in respiratory motion artifacts that blur anatomical boundaries and corrupt deep learning model predictions.
While previous medical imaging benchmarks have focused extensively on scanner vendor variability or anatomical diversity, the impact of unexpected human breathing behaviors has remained largely unaddressed. This paper summarizes the design and results of the MICCAI CMRxMotion challenge, which explicitly tasks machine learning models with evaluating image quality and performing robust segmentation under varying degrees of respiratory motion.
Methods
The challenge was divided into two distinct tasks:
- Automated Image Quality Assessment (IQA): Teams developed models to classify 3D CMR volumes into three categories: mild, intermediate, or severe motion. The winning approach (
A1. UON_IMA) leveraged a hierarchical ensemble of 2D CNNs (ResNet, EfficientNet) and Vision Transformers (ViT) pre-trained on ImageNet, utilizing both raw intensity images and 2D in-plane gradient magnitude maps. - Robust Cardiac Segmentation (RCS): Algorithms delineated the left ventricle (LV), right ventricle (RV), and myocardium (MYO) in the presence of motion. The top spot resulted in a tie between
S1. UA-SVCC(a modified nnU-Net using a multi-stage training progression from cross-entropy to Dice loss) andS2. Med-Air(an ensemble of nnU-Net, Swin-UNETR, and Swin-UNet utilizing adversarial data augmentation via the AdvChain framework).
Datasets
The authors prospectively curated a unique dataset of 320 3D CMR cine volumes from 40 healthy volunteers. To isolate respiratory motion from confounding factors like scanner vendor or underlying pathology, each volunteer underwent four separate scans on the same 3T scanner during a single visit. Each scan used a different breathing protocol: standard breath-hold, half breath-hold, free breathing, and intensive breathing.
Slices were extracted at the end-diastolic (ED) and end-systolic (ES) phases. The dataset was partitioned at the subject level into a training set (160 volumes), a validation set (40 volumes), and a fully withheld test set (120 volumes).
Key Insights and Ideas
- 2D and Lightweight Models Outperformed 3D: Interestingly, 2D slice-wise models generally outperformed their 3D counterparts, and lightweight backbones proved highly competitive. This is likely because smaller models face a reduced risk of overfitting on relatively small medical datasets.
- The Intermediate-to-Severe Blindspot: While models were highly accurate at identifying pristine, high-quality images (mild motion), they struggled significantly to differentiate between intermediate and severe motion artifacts. Fine-grained artifact assessment remains an open research problem.
- The Downstream Domino Effect: Subtle segmentation errors caused by motion artifacts don't just lower validation scores; they exponentially amplify errors in critical clinical biomarkers like ejection fraction and ventricular volumes, which could directly impact patient management and diagnosis.
Key Results
- Quality Assessment: For the IQA task, the winning team achieved a Cohen’s Kappa ($\kappa$) of 0.631 and an overall accuracy of 0.725.
- Segmentation Accuracy: For the segmentation task, the top models achieved impressive average Dice scores across the full test set: up to 93.90% for the LV, 92.39% for the RV, and 87.38% for the complex, thin-walled myocardium.
- Impact of Motion: Subgroup analysis revealed that as motion severity increased from mild to severe, the top algorithm's LV segmentation Dice score dropped by over 3 percentage points, while the Mean Absolute Error (MAE) for the derived Left Ventricular End-Diastolic Volume (LVEDV) more than doubled.
Limitations and Future Work
- Dataset Homogeneity: The dataset was collected exclusively from healthy volunteers at a single imaging center, meaning it lacks real-world clinical diversity and pathological variations.
- Computational Overhead: The most accurate segmentation models relied heavily on massive, multi-model ensembles. This makes them computationally intensive and slow—a major hurdle for real-time clinical deployment.
- Future Directions: The authors suggest exploring Vision Foundation Models, optimizing lightweight architectures for rapid inference, and developing synergistic frameworks that combine motion-robust segmentation directly with accelerated, motion-corrected MRI reconstruction at the source.
Summary updated