Abstract
Medical image segmentation typically demands extensive dense annotations for model training, which is both time-consuming and skill-intensive. To mitigate this burden, exemplar-based medical image segmentation methods have been introduced to achieve effective training with only one annotated image. In this paper, we introduce a novel Cross-model Mutual learning framework for Exemplar-based Medical image Segmentation (CMEMS), which leverages two models to mutually excavate implicit information from unlabeled data at multiple granularities. CMEMS can eliminate confirmation bias and enable collaborative training to learn complementary information by enforcing consistency at different granularities across models. Concretely, cross-model image perturbation based mutual learning is devised by using weakly perturbed images to generate high-confidence pseudo-labels, supervising predictions of strongly perturbed images across models. This approach enables joint pursuit of prediction consistency at the image granularity. Moreover, cross-model multi-level feature perturbation based mutual learning is designed by letting pseudo-labels supervise predictions from perturbed multi-level features with different resolutions, which can broaden the perturbation space and enhance the robustness of our framework. CMEMS is jointly trained using exemplar data, synthetic data, and unlabeled data in an end-to-end manner. Experimental results on two medical image datasets indicate that the proposed CMEMS outperforms the state-of-the-art segmentation methods with extremely limited supervision.
Introduction
Deep learning models for medical image segmentation are notoriously "data-hungry," typically requiring hundreds or thousands of pixel-perfect annotations. In the medical field, these labels are expensive and require expert knowledge. While "few-shot" learning exists, exemplar-based learning represents the extreme version: training with exactly one labeled image and a large pool of unlabeled data. Previous attempts, such as ELSNet, often suffered from confirmation bias, where a model's errors on unlabeled data are reinforced through self-training. The proposed CMEMS framework addresses this by using two models that keep each other in check through mutual learning.
Methodology
The core of CMEMS is a dual-model architecture where two identical networks ($N_1$ and $N_2$) are trained simultaneously. The framework integrates three primary strategies:
- Exemplar-based Data Synthesis: To initialize training, a "copy-and-paste" method ($\Omega$) extracts organs from the single annotated exemplar and pastes them onto various backgrounds with geometric and intensity transformations, creating a synthetic supervised dataset ($\mathcal{D}_S$).
- Cross-model Image Perturbation (CMIP): This strategy enforces consistency at the image granularity. One model processes a "weakly" perturbed version ($I^W$) of an unlabeled image to generate high-confidence pseudo-labels, which then supervise the predictions of the other model on a "strongly" perturbed version ($I^S$) of the same image.
- Cross-model Multi-level Feature Perturbation (CMFP): To enhance robustness, perturbations (e.g., channel dropping) are applied to internal feature maps at multiple resolutions within the encoder. Pseudo-labels from one model supervise the predictions derived from these perturbed features in the other model, broadening the perturbation space.
Experiments and Results
The researchers evaluated CMEMS on two benchmarks: the Synapse Multi-organ CT dataset and the Automated Cardiac Diagnosis Challenge (ACDC) dataset. In both cases, models were trained using only one annotated case.
- Synapse Dataset: CMEMS achieved a Dice Similarity Coefficient (DSC) of 0.597, nearly doubling the performance of the previous state-of-the-art, ELSNet (0.315). It also significantly reduced the Hausdorff Distance (HD95) from 109.7 to 55.02.
- ACDC Dataset: CMEMS reached a DSC of 0.817, compared to ELSNet's 0.410. Notably, CMEMS performance on ACDC (0.817) approached that of a fully supervised model (0.898) despite the extreme data scarcity.
Discussion and Conclusion
The success of CMEMS highlights that "two heads are better than one" in semi-supervised settings. By using two models to supervise each other across different perturbation levels, the framework effectively filters noise and prevents the reinforcement of errors. Enforcing consistency at both global (image) and local (multi-resolution feature) levels is vital for learning generalized semantic features. While the framework doubles computational overhead during training, it significantly closes the performance gap between exemplar-based and fully supervised learning. Future work may explore native 3D implementations and active learning for optimal exemplar selection.
Summary updated