Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding

Liyan Sun, Chenxin Li, Xinghao Ding, Yue Huang, Guisheng Wang, Yizhou Yu

This paper presents GCN-DE, a novel framework for few-shot medical image segmentation that leverages global spatial correlations and discriminative feature embeddings to generalize to new, unseen organs using only a handful of annotated examples. By combining an efficient global correlation module with a loss function that clusters similar anatomical structures, the authors achieve state-of-the-art results on both CT and MRI abdominal datasets.

TL;DR

GCN-DE addresses the scarcity of expert-annotated medical data by using a "few-shot" approach that learns to segment new organs from just one or two examples. It succeeds by capturing global spatial relationships between images and enforcing a feature space where different organs are clearly separated, significantly outperforming existing methods on abdominal CT and MRI scans.

What Problem Does This Paper Address?

Deep learning has revolutionized medical image segmentation, but it remains "data-hungry," typically requiring hundreds of expert-labeled scans to perform well. In clinical practice, getting these labels is expensive and time-consuming.

While "few-shot learning" (FSL) has been proposed to solve this—allowing models to learn from a "support" image and apply that knowledge to a "query" image—standard FSL methods designed for natural images (like dogs or cars) often fail in the medical domain. Medical images have unique properties:

  1. Spatial Regularity: Organs generally appear in the same relative positions (e.g., the liver is always near the right kidney).
  2. High Intra-class Consistency: A spleen looks very similar across different patients compared to the vast variety found in natural object classes.
  3. Spatial Non-alignment: Despite regularity, the exact position and shape of organs vary slice-by-slice, making simple local matching ineffective.

As shown in Figure 1, the challenge is to bridge the connection between these spatially distant but semantically identical objects.

Figure 1

Methods

The authors propose a dual-branch architecture (Support and Query) trained via episodic training. The core innovations are:

  • Global Correlation (GC) Module: Standard methods often use "prototypes" (averaging features into a single vector), which discards shape information. The GC module instead computes the relationship between every pixel in the support image and every pixel in the query image. To keep this computationally feasible, they decompose the operation into long-range and short-range spatial correlations, as detailed in Figure 3.
  • Discriminative Embedding (DE): To prevent the model from confusing similar-looking tissues, the authors introduce a DE loss. This forces the network to map features of the same organ (e.g., liver to liver) close together in latent space while pushing features of different organs (e.g., liver vs. spleen) far apart. This concept is illustrated in Figure 5.
  • Integrated Architecture: The full pipeline, shown in Figure 2, uses these modules alongside Spatial Squeeze-and-Excitation (sSE) blocks to refine the segmentation mask.
Figure 3
Figure 5
Figure 2

Datasets

The model was evaluated on two primary abdominal benchmarks:

  1. CHAOS (MRI): 20 3D T2-SPIR MRI scans with annotations for the liver, left/right kidneys, and spleen.
  2. MICCAI 2015 Multi-Atlas Abdomen Labeling (CT): 30 scans with the same four organ annotations.

The authors used a 5-fold cross-validation strategy, where one organ is held out as the "unseen" class during training to test the model's few-shot generalization capability.

Key Insights and Ideas

  • Shape Matters: By avoiding global average pooling (prototyping), the GC module preserves the morphological "signature" of organs, which is a critical cue in radiology.
  • Context is King: Medical backgrounds are not random noise; they are structured. The DE module leverages this by learning that the "neighborhood" of a kidney is a discriminative feature in itself.
  • Efficiency through Decomposition: Computing a full global correlation matrix is $O((HW)^2)$. By splitting this into long-range (sampled pixels) and short-range (local subregions), the authors achieve the benefits of non-local networks without the prohibitive memory cost.

Key Results

GCN-DE achieved the highest mean Dice Coefficient (DC) scores across both modalities, outperforming established few-shot models like PANet and SENet.

  • MRI Performance: Reached a mean DC of 67.30%, a significant jump over the next best competitor (co-FCN at 60.70%).
  • CT Performance: Reached a mean DC of 61.73%, nearly 20 percentage points higher than the baseline methods.
  • Qualitative Success: As seen in Figure 6, the DE module effectively suppresses false positives in regions where other organs reside, leading to much cleaner segmentation boundaries.
Figure 6

Limitations and Future Work

While the results are impressive, the paper focuses primarily on abdominal organs. The authors do not discuss how the model would handle more complex, diffuse structures like tumors or vascular networks, which lack the rigid spatial regularity of organs like the liver or kidneys. Additionally, the current implementation processes 2D slices; extending this to native 3D global correlation could further improve volumetric consistency but would require even more sophisticated computational optimizations.

Summary updated

Abstract

Despite deep convolutional neural networks achieved impressive progress in medical image computing and analysis, its paradigm of supervised learning demands a large number of annotations for training to avoid overfitting and achieving promising results. In clinical practices, massive semantic annotations are difficult to acquire in some conditions where specialized biomedical expert knowledge is required, and it is also a common condition where only few annotated classes are available. In this work, we proposed a novel method for few-shot medical image segmentation, which enables a segmentation model to fast generalize to an unseen class with few training images. We construct our few-shot image segmentor using a deep convolutional network trained episodically. Motivated by the spatial consistency and regularity in medical images, we developed an efficient global correlation module to capture the correlation between a support and query image and incorporate it into the deep network called global correlation network. Moreover, we enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class while keep the feature domains of different organs far apart. Ablation Study proved the effectiveness of the proposed global correlation module and discriminative embedding loss. Extensive experiments on anatomical abdomen images on both CT and MRI modalities are performed to demonstrate the state-of-the-art performance of our proposed model.
BibTeX
@article{sun2020few,
  author = {Liyan Sun and Chenxin Li and Xinghao Ding and Yue Huang and Guisheng Wang and Yizhou Yu},
  title = {{Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding}},
  journal = {Computers in Biology and Medicine},
  year = {2020},
  doi = {10.1016/j.compbiomed.2021.105067},
}
Figure