RobustEMD: Domain Robust Matching for Cross-domain Few-shot Medical Image Segmentation

Yazhou Zhu, Minxian Li, Qiaolin Ye, Shidong Wang, Tong Xin, Haofeng ZhangArtif. Intell. Medicine 2024 · DOI · OpenReview

TL;DR

This paper introduces RobustEMD, a novel domain-robust matching mechanism for few-shot medical image segmentation. By reformulating Earth Mover's Distance (EMD) to account for domain shifts and incorporating texture complexity for node weighting and boundary-aware cost functions, RobustEMD significantly improves segmentation performance across diverse medical imaging domains.

What Problem Does This Paper Address?

Medical image segmentation, especially when dealing with limited annotated data (few-shot learning), is crucial for diagnosis and treatment planning. However, existing few-shot medical image segmentation (FSMIS) models typically perform well only within the specific domain (e.g., imaging modality, scanner, institution) they were trained on. This is a significant limitation because real-world clinical data often comes from various sources, exhibiting domain shifts due to differences in image structure, texture, and acquisition protocols. This discrepancy leads to a performance drop when these models are applied to new, unseen domains. The paper addresses this critical gap by proposing a method that can generalize FSMIS models across these diverse medical imaging domains.

Methods

The core contribution of this work is the RobustEMD matching mechanism. It builds upon the Earth Mover's Distance (EMD) framework, which is typically used for comparing distributions. Here's a breakdown of the key novelties:

  • Domain-Robust EMD Formulation: Instead of directly applying EMD, RobustEMD reformulates it to specifically handle cross-domain challenges. It focuses on calculating correlation scores between foreground features from support and query images.
  • Texture Complexity Aware Node Weighting: The authors hypothesize that nodes with higher texture complexity are more domain-relevant. They introduce a method to generate node weights based on the local variance of Sobel-filtered image gradients. This effectively de-emphasizes domain-specific texture features during matching.
  • Boundary Aware Cost Function: Traditional EMD uses simple distance metrics (like L1 or L2 norms). RobustEMD introduces a novel cost function based on the Hausdorff distance. This metric is designed to preserve boundary information during the transportation process, leading to more accurate segmentation of organ edges.
  • Foreground Feature Generation: Before matching, foreground features are extracted from both support and query images using a ResNet backbone. This process involves using predicted masks to isolate relevant features.

Datasets

The paper evaluates RobustEMD on three cross-domain scenarios:

  • Cross-modal:
    • CHAOS dataset (MRI)
    • BTCV dataset (CT)
  • Cross-sequence:
    • LGE sequence cardiac MRI images
    • bSSFP sequence cardiac MRI images
  • Cross-institution:
    • Prostate MRI images from SmartTarget Biopsy Trial center
    • Prostate MRI images from TCIA Prostate3T center

The authors also mention using pseudo masks generated by a supervoxel clustering algorithm for self-supervised learning during training.

Key Insights and Ideas

  • Domain Shifts are a Major Hurdle for FSMIS: The paper clearly articulates that the real-world applicability of FSMIS is severely hampered by the inability of current models to generalize across different imaging domains.
  • Matching Mechanism is Key: The authors identify the matching mechanism as a critical bottleneck for cross-domain generalization. Their focus on improving this component is a strategic choice.
  • Texture Complexity as a Proxy for Domain Relevance: The insight that texture complexity can indicate domain-specific features, and thus should be down-weighted, is a clever way to encourage domain-agnostic feature learning.
  • Boundary Preservation is Crucial for Medical Segmentation: The introduction of a boundary-aware cost function highlights the importance of preserving fine details, especially in medical imaging where precise organ outlines are vital.

Key Results

RobustEMD demonstrates state-of-the-art (SoTA) performance across all tested cross-domain scenarios:

  • Quantitative Results:
    • On the cross-modal dataset, RobustEMD achieved the highest mean DSC scores, outperforming previous SoTA methods by significant margins (e.g., 3.66% and 9.64% improvements with ResNet50).
  • In the cross-sequence dataset, it also achieved the best performance on key segmentation regions like LV-BP, LV-MYO, and RV.
  • For the cross-institution dataset, RobustEMD consistently achieved the highest DSC scores for bladder, central gland, and rectum segmentation.
  • Qualitative Results: Visual comparisons showed that RobustEMD produced segmentations with better image structure and object boundary preservation, particularly for organs like the liver and kidneys, and cardiac structures.

Limitations and Future Work

The authors acknowledge a few limitations and suggest future research directions:

  • Reliance on Supervoxel Clustering: The current pipeline still relies on supervoxel clustering for generating pseudo masks for self-supervised learning. Future work could explore more accurate pseudo mask generation or even training models without self-supervised learning.
  • Exploring More Domain-Relevant Factors: The paper suggests investigating other categories of domain-relevant factors beyond initial image level or extracted features to further enhance domain generalization.

Summary updated

Abstract

Few-shot medical image segmentation (FSMIS) aims to perform the limited annotated data learning in the medical image analysis scope. Despite the progress has been achieved, current FSMIS models are all trained and deployed on the same data domain, as is not consistent with the clinical reality that medical imaging data is always across different data domains (e.g. imaging modalities, institutions and equipment sequences). How to enhance the FSMIS models to generalize well across the different specific medical imaging domains? In this paper, we focus on the matching mechanism of the few-shot semantic segmentation models and introduce an Earth Mover's Distance (EMD) calculation based domain robust matching mechanism for the cross-domain scenario. Specifically, we formulate the EMD transportation process between the foreground support-query features, the texture structure aware weights generation method, which proposes to perform the sobel based image gradient calculation over the nodes, is introduced in the EMD matching flow to restrain the domain relevant nodes. Besides, the point set level distance measurement metric is introduced to calculated the cost for the transportation from support set nodes to query set nodes. To evaluate the performance of our model, we conduct experiments on three scenarios (i.e., cross-modal, cross-sequence and cross-institution), which includes eight medical datasets and involves three body regions, and the results demonstrate that our model achieves the SoTA performance against the compared models.
BibTeX
@article{zhu2024robustemd,
  author = {Yazhou Zhu and Minxian Li and Qiaolin Ye and Shidong Wang and Tong Xin and Haofeng Zhang},
  title = {{RobustEMD: Domain Robust Matching for Cross-domain Few-shot Medical Image Segmentation}},
  journal = {Artif. Intell. Medicine},
  year = {2024},
  doi = {10.1016/j.artmed.2025.103197},
}
Figure