Quality-aware semi-supervised learning for CMR segmentation

Bram Ruijsink, Esther Puyol-Antón, Ye Li, Wenjia Bai, Eric Kerfoot, Reza Razavi, Andrew P. King

Abstract

One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of annotated training data. To overcome this limitation, data augmentation and semi-supervised learning (SSL) methods have been developed. However, these methods have limited effectiveness as they either exploit the existing data set only (data augmentation) or risk negative impact by adding poor training examples (SSL). Segmentations are rarely the final product of medical image analysis - they are typically used in downstream tasks to infer higher-order patterns to evaluate diseases. Clinicians take into account a wealth of prior knowledge on biophysics and physiology when evaluating image analysis results. We have used these clinical assessments in previous works to create robust quality-control (QC) classifiers for automated cardiac magnetic resonance (CMR) analysis. In this paper, we propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks, that are subsequently utilised for further network training. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks (semiQCSeg). We evaluate our approach in two CMR segmentation tasks (aortic and short axis cardiac volume segmentation) using UK Biobank data and two commonly used network architectures (U-net and a Fully Convolutional Network) and compare against supervised and SSL strategies. We show that semiQCSeg improves training of the segmentation networks. It decreases the need for labelled data, while outperforming the other methods in terms of Dice and clinical metrics. SemiQCSeg can be an efficient approach for training segmentation networks for medical image data when labelled datasets are scarce.

Introduction

Deep learning models for medical image segmentation, particularly in Cardiac Magnetic Resonance (CMR), are severely bottlenecked by the lack of high-quality labeled data. Manually segmenting a full cardiac cycle—which involves multiple slices across 30-50 timeframes—is an incredibly labor-intensive task for clinicians. Consequently, most training datasets are sparse, often containing annotations for only one or two frames per patient.

To mitigate this, researchers often turn to data augmentation or Semi-Supervised Learning (SSL). Standard SSL generates pseudo-labels for unlabelled data to expand the training set. However, this approach faces a significant "garbage in, garbage out" risk: if the network generates a poor segmentation, standard SSL blindly incorporates it, polluting the training pool and potentially degrading model performance. This paper introduces semiQCSeg, a framework designed to ensure only high-quality pseudo-labels are used for network refinement by bridging the gap between pixel-level segmentation and downstream clinical physiology.

Methods

The semiQCSeg framework utilizes an independent Quality Control (QC) classifier to filter machine-generated segmentations based on physiological plausibility. Unlike standard SSL, which might rely on internal network confidence scores, this method leverages domain knowledge of biological organ behavior.

The semiQCSeg framework workflow: from partly labeled data through a segmentation network and QC classifier to 'best-in-class' data augmentation.

The process follows five key steps:

  1. Initial Training: A standard segmentation network (e.g., FCN for the aorta or U-Net for ventricles) is trained on a small, manually labeled dataset.
  2. Pseudo-label Generation: The network predicts segmentations for unlabelled, full-cardiac-cycle data.
  3. Physiological Extraction: Biological metrics, such as aortic area or ventricular volume changes over the heartbeat, are calculated from these predictions.
  4. QC Filtering: These physiological curves are processed by a pre-trained 3-layer LSTM network. This LSTM identifies "impossible" or erratic volume changes, classifying segmentations as either 'Accept' or 'Revise/Reject'.
  5. Augmentation: The top "best-in-class" segmentations (e.g., the top 30) are added back into the training dataset for model retraining.

Experimental Setup

The study utilized data from the UK Biobank cohort across two primary tasks:

  • Aorta Dataset: 138 subjects for training (3 labeled frames each) and 102 subjects for testing (full-cycle ground truth).
  • Short-Axis (SAX) Dataset: 500 subjects for training (2 labeled frames each) and 102 subjects for testing.
  • QC Training: A separate set of 1,000 cases, evaluated by a clinician, was used to train the LSTM QC network, ensuring no data leakage into the segmentation tasks.

Results

Experimental results demonstrate that semiQCSeg consistently outperforms standard SSL and, in some cases, even exceeds the performance of models trained on much larger manually annotated datasets.

Scatter plots showing the change in Dice score for Aorta (A) and SAX (B) datasets when using the semiQCSeg method.

In the Aorta Segmentation task, semiQCSeg achieved a Dice score of 95.56, significantly higher than the standard SSL baseline (92.86) and matching the performance of a model trained on a full dataset (95.45). For SAX Segmentation, the method achieved a Dice score of 89.44, crushing both the half-dataset baseline (82.57) and standard SSL (85.30). Notably, it outperformed the full-dataset model (88.17), likely because the high-quality, full-cycle pseudo-labels provided richer temporal data than the sparse manual annotations.

Clinical metrics further validated the approach. When calculating the Left Ventricular Ejection Fraction (LVEF), semiQCSeg yielded results (57 ± 6) that were the most aligned with the manual ground truth (58 ± 5).

Discussion and Conclusion

The primary innovation of semiQCSeg is the use of physiology as a filter. By mimicking how a clinician evaluates an MRI—looking for physically possible movement rather than just pixel patterns—the model safely leverages unlabelled data.

While the authors note a potential for bias if the network only learns "easy" cases, this was not observed in their experiments. Future work could involve extending this framework to time-sensitive architectures like LSTMs or applying it to other modalities like CT imaging. Overall, semiQCSeg represents a significant step toward efficient, high-performance medical image segmentation in data-scarce environments.

Summary updated

Abstract

One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of annotated training data. To overcome this limitation, data augmentation and semi-supervised learning (SSL) methods have been developed. However, these methods have limited effectiveness as they either exploit the existing data set only (data augmentation) or risk negative impact by adding poor training examples (SSL). Segmentations are rarely the final product of medical image analysis - they are typically used in downstream tasks to infer higher-order patterns to evaluate diseases. Clinicians take into account a wealth of prior knowledge on biophysics and physiology when evaluating image analysis results. We have used these clinical assessments in previous works to create robust quality-control (QC) classifiers for automated cardiac magnetic resonance (CMR) analysis. In this paper, we propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks, that are subsequently utilised for further network training. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks (semiQCSeg). We evaluate our approach in two CMR segmentation tasks (aortic and short axis cardiac volume segmentation) using UK Biobank data and two commonly used network architectures (U-net and a Fully Convolutional Network) and compare against supervised and SSL strategies. We show that semiQCSeg improves training of the segmentation networks. It decreases the need for labelled data, while outperforming the other methods in terms of Dice and clinical metrics. SemiQCSeg can be an efficient approach for training segmentation networks for medical image data when labelled datasets are scarce.
BibTeX
@inproceedings{ruijsink2020quality,
  author = {Bram Ruijsink and Esther Puyol-Antón and Ye Li and Wenjia Bai and Eric Kerfoot and Reza Razavi and Andrew P. King},
  title = {{Quality-aware semi-supervised learning for CMR segmentation}},
  booktitle = {M\&Ms and EMIDEC/STACOM @ MICCAI},
  year = {2020},
  url = {https://openreview.net/forum?id=A6a1o1keKxzb},
}
Figure