CrossMatch: Enhance Semi-Supervised Medical Image Segmentation with Perturbation Strategies and Knowledge Distillation

Bin Zhao, Chunshi Wang, Shuxue Ding

Abstract

Semi-supervised learning for medical image segmentation presents a unique challenge of efficiently using limited labeled data while leveraging abundant unlabeled data. Despite advancements, existing methods often do not fully exploit the potential of the unlabeled data for enhancing model robustness and accuracy. In this paper, we introduce CrossMatch, a novel framework that integrates knowledge distillation with dual perturbation strategies—image-level and feature-level—to improve the model's learning from both labeled and unlabeled data. CrossMatch employs multiple encoders and decoders to generate diverse data streams, which undergo self-knowledge distillation to enhance consistency and reliability of predictions across varied perturbations. Our method significantly surpasses other state-of-the-art techniques in standard benchmarks by effectively minimizing the gap between training on labeled and unlabeled data and improving edge accuracy and generalization in medical image segmentation. The efficacy of CrossMatch is demonstrated through extensive experimental validations, showing remarkable performance improvements without increasing computational costs.

Introduction

In medical imaging, high-quality pixel-level annotations for 3D CT or MRI scans are incredibly expensive and time-consuming to produce, requiring expert radiologists' labor. Semi-supervised learning (SSL) aims to solve this by training models on a small set of labeled data and a much larger pool of unlabeled data.

The core challenge in SSL is "consistency regularization"—ensuring the model produces stable predictions even when the input is slightly perturbed. While many methods use image-level perturbations (like rotating or flipping an image), they often fail to fully exploit feature-level perturbations (noise injected deep within the neural network). Furthermore, traditional knowledge distillation (KD) usually requires a heavy, pre-trained "teacher" model to guide a "student," which adds significant computational overhead.

Comparison of different knowledge distillation and semi-supervised learning paradigms.

Methods

The authors introduce CrossMatch, a single-stage framework that treats different internal "data streams" as teachers and students, moving beyond conventional KD or simple FixMatch approaches.

Dual Perturbation Strategy

The model applies two layers of noise. First, it uses weak and strong image-level augmentations. Second, it injects feature-level perturbations (using Dropout3D) at the bottleneck of the encoder-decoder architecture. This dual approach forces the model to learn robust representations that are invariant to both visual and latent noise.

Multi-Stream Architecture and Self-Distillation

CrossMatch employs two encoders (weak vs. strong image perturbation) and three decoders (no dropout, weak dropout, and strong dropout). This creates six distinct output streams from a single unlabeled input. The stream with the least amount of noise (weak image augmentation + no feature dropout) is designated as the Teacher. Its high-confidence predictions serve as pseudo-labels to train all other "Student" streams.

Detailed architecture of the CrossMatch framework showing the interaction between encoders, decoders, and distillation losses.

To further boost stability, the authors implement Decoder Distillation. This helps the model bridge the "capability gap" between different levels of perturbation. Notably, the authors found that for medical imaging, using Dice loss for distillation is superior to standard KL divergence because it better handles the extreme class imbalance typical of organ segmentation.

Experimental Results

The framework was evaluated on four diverse benchmarks: 2018 Left Atrium (3D MRI), ACDC (MRI), Pancreas-CT (3D CT), and ISIC 2018 (2D skin lesions). CrossMatch consistently set new state-of-the-art (SOTA) records:

  • Surpassing Fully Supervised Baselines: On the LA dataset, CrossMatch trained with only 10% labeled data achieved a Dice score of 91.33%, surpassing a standard V-Net trained on 100% of the labels (91.14%).
  • Low-Data Robustness: On the ACDC dataset with only 3 labeled scans (5%), CrossMatch improved the Dice score by nearly 4% over the previous best method, UniMatch.
  • Efficiency: Despite the multiple logical streams, clever batch-stacking allows CrossMatch to run faster (210ms per iteration) than many competing SSL frameworks like MC-Net or CAML.

Discussion and Conclusion

CrossMatch demonstrates that by carefully controlling the "perturbation space," a model can effectively teach itself without the need for a separate, heavy teacher model. Qualitative results show smoother segmentation boundaries and fewer misclassified "islands" compared to other methods.

While efficient, the model still requires multiple forward passes per iteration, which can be taxing on GPU memory for high-resolution 3D volumes. Future work may involve integrating shape-constraint models or active contour loss to help the model better understand the underlying anatomy of difficult regions, such as the tail of the pancreas or atrial appendages.

Summary updated

Abstract

Semi-supervised learning for medical image segmentation presents a unique challenge of efficiently using limited labeled data while leveraging abundant unlabeled data. Despite advancements, existing methods often do not fully exploit the potential of the unlabeled data for enhancing model robustness and accuracy. In this paper, we introduce CrossMatch, a novel framework that integrates knowledge distillation with dual perturbation strategies-image-level and feature-level-to improve the model's learning from both labeled and unlabeled data. CrossMatch employs multiple encoders and decoders to generate diverse data streams, which undergo self-knowledge distillation to enhance consistency and reliability of predictions across varied perturbations. Our method significantly surpasses other state-of-the-art techniques in standard benchmarks by effectively minimizing the gap between training on labeled and unlabeled data and improving edge accuracy and generalization in medical image segmentation. The efficacy of CrossMatch is demonstrated through extensive experimental validations, showing remarkable performance improvements without increasing computational costs. Code for this implementation is made available at https://github.com/AiEson/CrossMatch.git.
BibTeX
@article{zhao2025crossmatch,
  author = {Bin Zhao and Chunshi Wang and Shuxue Ding},
  title = {{CrossMatch: Enhance Semi-Supervised Medical Image Segmentation with Perturbation Strategies and Knowledge Distillation}},
  journal = {IEEE JBHI},
  year = {2025},
  url = {https://openreview.net/forum?id=lhnrePaSZI},
}
Figure