Rethinking Barely-Supervised Segmentation from an Unsupervised Domain Adaptation Perspective

Zhiqiang Shen, Peng Cao, Junming Su, Jinzhu Yang, Osmar R. ZaïanearXiv 2024 · OpenReview

TL;DR

This paper tackles the challenging problem of barely-supervised volumetric medical image segmentation (BSS), where only a tiny fraction of data is labeled (single slices). Instead of relying on noisy pseudo-labels generated by traditional registration methods, the authors reframe BSS as an unsupervised domain adaptation (UDA) problem. They introduce a novel framework, BvA, which uses a noise-free data construction method and a frequency/spatial Mix-Up strategy to achieve state-of-the-art results with significantly less annotation.

What Problem Does This Paper Address?

Medical image segmentation is crucial for diagnosis and treatment planning, but annotating volumetric medical images (like MRIs or CT scans) is incredibly time-consuming and requires expert knowledge. This is especially true for "barely-supervised segmentation" (BSS), where the available labeled data is extremely scarce – typically consisting of a few images, each with only a single slice annotated.

Existing BSS methods often use a "registration-based paradigm." This involves using image registration techniques to "propagate" the single-slice annotation across the entire volume, creating pseudo-labels for the whole 3D image. These pseudo-labeled volumes are then used in a semi-supervised learning setup. However, this approach suffers from a critical flaw: the registration process is prone to errors, leading to noisy and unreliable pseudo-labels. This noise significantly degrades the segmentation performance, especially when the initial annotation is minimal. This paper aims to overcome this limitation by proposing a new perspective and a more robust method.

Methods

The core of the proposed solution, BvA (Barely-supervised learning via unsupervised domain Adaptation), reframes BSS as an Unsupervised Domain Adaptation (UDA) problem. This means treating the synthesized, fully-labeled volumes as the "source domain" and the original, unlabeled medical images as the "target domain."

BvA consists of two main novel components:

  1. Noise-Free Labeled Data Construction (NFC): This algorithm synthesizes complete volumetric image-label pairs from single-slice annotations without relying on inter-slice registration. It works by:
  2. Dividing the single annotated slice into smaller patches.
  3. Stacking these patches to form a "cropped volume."

    • Reshaping this cropped volume to match the dimensions of the original volumetric image. The intuition is that inter-patch similarity within a slice mirrors inter-slice continuity in a volume.
  4. Frequency and Spatial Mix-Up (FSX): To bridge the domain gap between the synthesized (NFC-generated) images and the original medical images, FSX applies perturbations. It assumes that a well-generalized model should be robust to small changes in image style and content.

  5. Frequency Mix-Up: Perturbs the style of images by mixing amplitude components in the frequency domain, while preserving content.
  6. Spatial Mix-Up: Perturbs content by mixing image regions using a CutMix-like operation, applied simultaneously to images and their corresponding labels.

BvA also leverages a mean-teacher paradigm for training, using both the synthesized labeled data and the original unlabeled data.

Datasets

The proposed method was evaluated on two publicly available medical image segmentation datasets:

  • Left Atrial Segmentation 2018 (LA): Used for cardiac MRI segmentation.
  • Brain Tumor Segmentation 2020 (BraTS): A challenging dataset for segmenting brain tumors from multi-modal MRI scans.

The paper uses standard train/validation/test splits for these datasets.

Key Insights and Ideas

  • Reframing BSS as UDA: The most significant insight is shifting the problem formulation from semi-supervised learning with noisy pseudo-labels to unsupervised domain adaptation. This allows leveraging powerful UDA techniques to handle domain shifts directly, rather than trying to clean up registration artifacts.
  • NFC's Novelty: Generating volumetric labels without registration is a key innovation. It bypasses the primary source of noise in existing methods and directly addresses the "slice-to-volume" challenge.
  • FSX for Domain Bridging: The combination of frequency and spatial Mix-Up is a clever way to tackle both style (frequency) and content (spatial) domain shifts, making the synthesized data more aligned with real-world medical images.
  • Annotation Strategy Matters: The study highlights that for volumetric segmentation, annotating multiple images with single slices is more effective than annotating multiple slices within a single image, especially for organs with high inter-slice redundancy like the left atrium.

Key Results

  • State-of-the-Art Performance: BvA significantly outperforms existing barely-supervised segmentation methods (PLN, DeSCO) on both LA and BraTS datasets.
  • Dramatic Improvement with Minimal Data: On the LA dataset with only one barely-labeled image (single slice), BvA achieved a Dice score of 81.20%, outperforming the state-of-the-art by a remarkable 61.71%.
  • Robustness: BvA shows substantial gains even with 5% and 10% barely-labeled data, consistently outperforming registration-based methods.
  • Qualitative Improvements: Visualizations show BvA producing smoother and more accurate segmentations compared to baseline methods, especially for complex structures.
  • Ablation Study Validation: Each component of BvA (NFC, FX, SX) contributes positively to the overall performance, demonstrating their effectiveness.

Limitations and Future Work

  • Hyperparameter Sensitivity: While the paper investigates hyperparameters like window size and stride for NFC, their optimal selection might still require careful tuning for different datasets.
  • Generalizability: While tested on two datasets, further validation on a wider variety of medical imaging modalities and anatomical structures would be beneficial.
  • Computational Cost: The Mix-Up strategies and UDA framework might introduce additional computational overhead compared to simpler methods, although the paper does not explicitly detail this.

Future work could explore more advanced UDA techniques, investigate the impact of different noise models in NFC, and potentially extend the framework to other weakly-supervised settings. The authors also suggest that their findings on annotation strategies could inform future data collection efforts.

Summary updated

Abstract

This paper investigates an extremely challenging problem: barely-supervised volumetric medical image segmentation (BSS). A BSS training dataset consists of two parts: 1) a barely-annotated labeled set, where each labeled image contains only a single-slice annotation, and 2) an unlabeled set comprising numerous unlabeled volumetric images. State-of-the-art BSS methods employ a registration-based paradigm, which uses inter-slice image registration to propagate single-slice annotations into volumetric pseudo labels, constructing a completely annotated labeled set, to which a semi-supervised segmentation scheme can be applied. However, the paradigm has a critical limitation: the pseudo-labels generated by image registration are unreliable and noisy. Motivated by this, we propose a new perspective: instead of solving BSS within a semi-supervised learning scheme, this work formulates BSS as an unsupervised domain adaptation problem. To this end, we propose a novel BSS framework, \textbf{B}arely-supervised learning \textbf{via} unsupervised domain \textbf{A}daptation (BvA), as an alternative to the dominant registration paradigm. Specifically, we first design a novel noise-free labeled data construction algorithm (NFC) for slice-to-volume labeled data synthesis. Then, we introduce a frequency and spatial Mix-Up strategy (FSX) to mitigate the domain shifts. Extensive experiments demonstrate that our method provides a promising alternative for BSS. Remarkably, the proposed method, trained on the left atrial segmentation dataset with \textbf{only one} barely-labeled image, achieves a Dice score of 81.20%, outperforming the state-of-the-art by 61.71%. The code is available at https://github.com/Senyh/BvA.
BibTeX
@misc{shen2024rethinking,
  author = {Zhiqiang Shen and Peng Cao and Junming Su and Jinzhu Yang and Osmar R. Zaïane},
  title = {{Rethinking Barely-Supervised Segmentation from an Unsupervised Domain Adaptation Perspective}},
  year = {2024},
  eprint = {2405.09777},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.09777},
}
Figure