Multi-task Swin Transformer for Motion Artifacts Classification and Cardiac Magnetic Resonance Image Segmentation

Michal K. Grzeszczyk, Szymon Plotka, Arkadiusz Sitek

Introduction

Cardiac Magnetic Resonance (CMR) imaging is the gold standard for assessing heart anatomy and function. However, the acquisition process is inherently slow, making it susceptible to motion artifacts caused by patient breathing, physical movement, or blood flow. These artifacts blur images and complicate the accurate segmentation of the left ventricle (LV), right ventricle (RV), and myocardium (MYO). Traditionally, image quality assessment and segmentation are treated as separate tasks. This paper introduces a unified Multi-task Swin UNETR architecture designed to simultaneously classify motion artifact severity and perform cardiac segmentation in a single forward pass.

Methodology

The researchers proposed a multi-task deep learning framework that leverages the strengths of transformer-based architectures for medical imaging.

The Swin UNETR Backbone

Unlike standard Convolutional Neural Networks (CNNs), the model uses a Swin Transformer as the encoder. Transformers are particularly effective at capturing "long-range dependencies," allowing the model to understand the global context of the cardiac anatomy better than local-receptive-field CNNs. This global view is essential when dealing with artifacts that may distort large portions of the image.

Multi-task Architecture

The model architecture splits into two distinct "heads" following the shared encoder:

  1. Segmentation Head: A U-Net-style convolutional decoder that generates high-resolution masks for the LV, RV, and MYO.
  2. Classification Head: A branch utilizing Global Average Pooling and fully-connected layers to categorize the image into three motion levels: mild, intermediate, or severe.

Training Strategy

To optimize the model, the team used a combined loss function (Cross-Entropy and DICE for segmentation; Cross-Entropy for classification). A key feature of the training protocol was selective learning: the model skipped segmentation training for images labeled with "severe" artifacts, as these are considered non-diagnostic in clinical practice. The model was validated using 5-fold stratified cross-validation to ensure stability.

Experimental Setup and Dataset

The study utilized the CMRxMotion challenge dataset (MICCAI 2022), which includes data from 45 volunteers. To simulate real-world variability, images were captured under four breathing conditions: full breath-hold, half breath-hold, free breathing, and intensive breathing. Experts categorized these images into three classes of motion severity, providing ground-truth segmentations only for the mild and intermediate classes.

Results

The model was evaluated on its dual ability to delineate heart structures and assess image quality:

  • Segmentation Accuracy: The ensemble achieved a DICE coefficient of 0.871 and a Hausdorff Distance (HD95) of 5.169, indicating high precision in identifying cardiac structures despite the presence of motion.
  • Classification Accuracy: The model reached an average accuracy of 0.595 and a Cohen’s Kappa of 0.184.

The researchers noted that classification was significantly more challenging than segmentation. This was primarily due to a severe dataset imbalance, with only 21 samples of "severe" motion compared to 70 "mild" samples, limiting the model's ability to learn the features of high-severity artifacts.

Discussion and Clinical Relevance

The multi-task approach offers two primary benefits. First, it is computationally efficient, requiring a lower footprint than running two independent networks. Second, Multi-task Learning (MTL) encourages representation sharing; by learning to identify artifacts, the encoder becomes more robust at finding anatomical edges in noisy data.

Clinically, this model mimics the standard medical workflow where a technician must first verify if a scan is of diagnostic quality before proceeding with measurements. Integrating such a model into MRI consoles could provide real-time feedback, allowing for immediate re-scans if motion artifacts are detected.

Conclusion and Future Work

While the Multi-task Swin UNETR demonstrates the potential for automated CMR quality control and analysis, the study highlights the need for larger, more balanced datasets. Future work will focus on incorporating data augmentation from other major challenges (such as the ACDC dataset), refining the smoothness of segmentation masks, and moving toward real-time clinical integration.

Summary updated

BibTeX
@inproceedings{grzeszczyk2022multi,
  author = {Michal K. Grzeszczyk and Szymon Plotka and Arkadiusz Sitek},
  title = {{Multi-task Swin Transformer for Motion Artifacts Classification and Cardiac Magnetic Resonance Image Segmentation}},
  booktitle = {STACOM @ MICCAI},
  year = {2022},
  doi = {10.1007/978-3-031-23443-9\_38},
}
Figure