Quality-Aware Semi-supervised Learning for CMR Segmentation

Bram Ruijsink, Esther Puyol-Antón, Ye Li, Wenjia Bai, Eric Kerfoot, Reza Razavi, Andrew P. King

TL;DR

Deep learning for medical image segmentation is severely bottlenecked by a lack of labelled data. This paper introduces semiQCSeg, a novel semi-supervised learning approach that uses an independent Quality Control (QC) network—based on physiological downstream tasks—to filter and select only the best machine-generated segmentations to add back into the training pool. This method significantly boosts model performance, even beating fully supervised models trained on larger datasets, while requiring far less manual annotation.

What Problem Does This Paper Address?

Training accurate deep learning models for Cardiac Magnetic Resonance (CMR) segmentation requires massive amounts of annotated data. However, manually segmenting a full cardiac cycle (multiple slices across 30-50 timeframes) is incredibly labor-intensive for clinicians. Consequently, training datasets usually only contain annotations for one or two frames per patient.

To fix this, researchers typically use data augmentation or Semi-Supervised Learning (SSL). Standard SSL generates pseudo-labels for unlabelled data and adds them to the training set. However, if the network generates a bad segmentation, standard SSL blindly adds it to the training pool anyway, polluting the data and degrading model performance. The authors address this by asking: How can we ensure only high-quality pseudo-labels are used to teach the network?

Methods

The authors propose semiQCSeg, a framework that bridges the gap between pixel-level segmentation and downstream clinical physiology.

Instead of relying on the segmentation network's own confidence scores, they use an independent Quality Control (QC) classifier.

  1. Initial Training: A standard segmentation network (FCN for the aorta, U-Net for the heart ventricles) is trained on a small, manually labelled dataset.
  2. Pseudo-label Generation: The network predicts segmentations for unlabelled, full-cardiac-cycle data.
  3. Physiological Extraction: The system calculates the actual biological metrics from these segmentations (e.g., aortic area or ventricular volume changes over the heartbeat).
  4. QC Filtering: These physiological curves are passed through a pre-trained 3-layer LSTM network. Because biological organs follow specific physical rules, this LSTM can easily spot "impossible" or erratic volume changes, classifying the segmentation as 'good' or 'erroneous'.
  5. Augmentation: The top 30 "best-in-class" segmentations are added back into the training dataset, and the segmentation network is retrained.

Datasets

The study utilizes data from the UK Biobank cohort:

  • Aorta Dataset: Single slice, 100 temporal frames. (Train: 138 subjects with 3 labelled frames each; Test: 102 subjects with full-cycle ground truth).
  • Short-Axis (SAX) Dataset: Multiple slices, 50 temporal frames for Left/Right Ventricle and Myocardium. (Train: 500 subjects with 2 labelled frames each; Test: 102 subjects with full-cycle ground truth).
  • QC Training Data: 1,000 separate cases evaluated by a clinician to train the LSTM. These were strictly isolated from the segmentation training/testing sets to prevent data leakage.

Key Insights and Ideas

  • Physiology as a Filter: Using downstream clinical metrics (volume curves) is a brilliant way to inject domain knowledge into the training loop. It mimics how a human doctor evaluates an MRI—they don't just look at pixels; they look at whether the heart is pumping in a physically possible way.
  • Quality over Quantity: Standard SSL often falls into the "garbage in, garbage out" trap. By strictly gating what gets added back into the training set, the model safely leverages unlabelled data.
  • Beating the "Upper Bound": Surprisingly, semiQCSeg actually outperformed a model trained on the full manually annotated dataset for the SAX task. Because manual labels are sparse (only 2 frames per patient), the high-quality, full-cycle pseudo-labels generated by semiQCSeg provided richer temporal data than the human annotators did.

Key Results

  • Aorta Segmentation (FCN): semiQCSeg achieved a Dice score of 95.56, beating the half-dataset baseline (93.71) and standard SSL (92.86), while matching the performance of the full-dataset model (95.45).
  • SAX Segmentation (U-Net): semiQCSeg achieved a Dice score of 89.44, crushing the half-dataset baseline (82.57) and standard SSL (85.30). Impressively, it outperformed the full-dataset model (88.17).
  • Clinical Metrics: When calculating downstream tasks like Left Ventricular Ejection Fraction (LVEF), semiQCSeg yielded results (57 ± 6) that were the closest of any model to the manual ground truth (58 ± 5).

Limitations and Future Work

  • Potential for Bias: The authors acknowledge that only selecting "good" segmentations might bias the network toward learning "easy" cases while ignoring complex pathologies. While they didn't observe this bias in their single-iteration experiment, they warn that running this loop for multiple iterations could cause issues. They suggest monitoring validation loss to trigger early stopping.
  • Upfront Cost: Training the QC network requires a clinician to manually review 1,000 cases to establish the baseline of what a "good" volume curve looks like.
  • Future Directions: The authors suggest extending this framework to time-sensitive segmentation architectures (like LSTMs) and applying it to other modalities, such as CT multi-organ segmentation or brain imaging, where multiple acquisitions can be exploited for quality control.

Summary updated

Abstract

One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of annotated training data. To overcome this limitation, data augmentation and semi-supervised learning (SSL) methods have been developed. However, these methods have limited effectiveness as they either exploit the existing data set only (data augmentation) or risk negative impact by adding poor training examples (SSL). Segmentations are rarely the final product of medical image analysis - they are typically used in downstream tasks to infer higher-order patterns to evaluate diseases. Clinicians take into account a wealth of prior knowledge on biophysics and physiology when evaluating image analysis results. We have used these clinical assessments in previous works to create robust quality-control (QC) classifiers for automated cardiac magnetic resonance (CMR) analysis. In this paper, we propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks, that are subsequently utilised for further network training. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks (semiQCSeg). We evaluate our approach in two CMR segmentation tasks (aortic and short axis cardiac volume segmentation) using UK Biobank data and two commonly used network architectures (U-net and a Fully Convolutional Network) and compare against supervised and SSL strategies. We show that semiQCSeg improves training of the segmentation networks. It decreases the need for labelled data, while outperforming the other methods in terms of Dice and clinical metrics. SemiQCSeg can be an efficient approach for training segmentation networks for medical image data when labelled datasets are scarce.
BibTeX
@inproceedings{ruijsink2020quality,
  author = {Bram Ruijsink and Esther Puyol-Antón and Ye Li and Wenjia Bai and Eric Kerfoot and Reza Razavi and Andrew P. King},
  title = {{Quality-Aware Semi-supervised Learning for CMR Segmentation}},
  booktitle = {M\&Ms and EMIDEC/STACOM @ MICCAI},
  year = {2020},
  url = {https://openreview.net/forum?id=A6a1o1keKxzb},
}

Loading related content...

Figure