Abstract
Few-shot medical image segmentation (FSS) is challenged by the scarcity of labeled data and the inherent complexity of anatomical structures. Traditional prototype-based methods often struggle to balance local detail with global context, frequently failing to adapt to unseen tasks at test time. This paper introduces a novel perspective by framing prototype generation as a Vector Quantization (VQ) problem. We propose a framework comprising three core components: Grid-Format VQ (GFVQ), Self-Organized VQ (SOVQ), and Residual Oriented VQ (ROVQ). These mechanisms collectively enhance local detail representation, enable adaptive feature clustering, and facilitate parameter-free task adaptation. Our approach achieves state-of-the-art performance across multiple MRI benchmarks, demonstrating significant improvements in segmentation accuracy and generalizability.
Introduction
Medical image segmentation is a cornerstone of modern clinical diagnosis, yet the reliance of deep learning models on massive, manually labeled datasets remains a significant bottleneck. Few-shot segmentation (FSS) addresses this by attempting to segment new organs or lesions using only a handful of labeled examples (the "support" set).
Existing FSS methods typically extract "prototypes"—representative feature vectors—from support images to guide the segmentation of query images. However, these methods face a persistent "Goldilocks" problem: a small number of prototypes cannot capture complex anatomical nuances, while an excessive number introduces noise and misclassification. Furthermore, most models lack the flexibility to adapt to the specific, often noisy characteristics of unseen medical tasks encountered during inference.
Methodology: The VQ Framework
We reimagine prototype generation as a feature space partitioning problem using Vector Quantization. By treating prototypes as "codebook" vectors that define decision boundaries in a high-dimensional space, we can more effectively represent the underlying data distribution. Our framework introduces three specialized VQ modules:
- Grid-Format VQ (GFVQ): Serving as a baseline for local detail, GFVQ divides the feature map into a rigid grid and averages features within each cell. This ensures a uniform quantization across the entire spatial extent, preserving basic structural information.
- Self-Organized VQ (SOVQ): Inspired by Self-Organizing Maps (SOM), SOVQ adaptively clusters feature points. Unlike the rigid grid of GFVQ, SOVQ allows "neurons" to compete to represent clusters of features. This adaptive clustering provides a "global view" of the feature distribution, enabling the model to better capture irregular shapes and sharp organ boundaries.
- Residual Oriented VQ (ROVQ): To address the adaptation gap, ROVQ fine-tunes learned prototypes using residual information from the support image. This is a parameter-free process occurring during the forward pass, allowing the model to "adjust" to specific task characteristics without requiring additional training or backpropagation.
Experimental Evaluation
The proposed framework was evaluated on three diverse MRI datasets: Abdomen MRI (CHAOS challenge), Cardiac MRI (MSCMR challenge), and Prostate MRI. These datasets represent a wide range of anatomical structures and imaging challenges.
Key Results
Our method achieved state-of-the-art (SOTA) results across all benchmarks, with performance gains becoming even more significant when combined with Self-Supervised Learning (SSL) pre-training:
- Abdomen MRI: Achieved a mean Dice score of 84.56, outperforming the previous best (SSL-ALP) by nearly 6 percentage points.
- Cardiac MRI: Reached a mean Dice score of 84.86, a substantial improvement over the previous SOTA of 76.90.
- Prostate MRI: Achieved a mean Dice score of 63.22, demonstrating robustness even on highly irregular and challenging structures.
Qualitative analysis shows that the dual-structure approach (GFVQ + SOVQ) effectively balances local structure with adaptive shape fitting, significantly reducing false positives compared to standard grid-based methods.
Discussion and Conclusion
This work demonstrates that shifting the perspective of prototypes from simple averages to support vectors in a VQ framework significantly enhances few-shot medical segmentation. While the iterative nature of SOVQ adds some computational overhead during inference, the gains in boundary precision and task adaptation are substantial.
Future research will explore integrating the self-organizing mechanism directly into the feature extraction backbone. This could potentially enable even more powerful unsupervised representation learning, further reducing the dependency on labeled data in clinical environments.
Summary updated