Abstract
Few-shot segmentation (FSS) aims to segment unseen-class objects given only a handful of densely labeled samples. Existing approaches typically follow the prototype learning paradigm to perform meta-inference, which fails to fully exploit the underlying information from support image-mask pairs, resulting in various segmentation failures, such as incomplete objects, ambiguous boundaries, and distractor activation. To this end, we propose a simple yet versatile framework in the spirit of divide-and-conquer. Specifically, a novel self-reasoning scheme is first implemented on the annotated support image, and then the coarse segmentation mask is divided into multiple regions with different properties. Leveraging effective masked average pooling operations, a series of support-induced proxies are derived, each playing a specific role in conquering the above challenges. Moreover, we devise a unique parallel decoder structure that integrates proxies with similar attributes to boost the discrimination power. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information as a guide at the "episode" level, not just about the object cues themselves. Extensive experiments on PASCAL-5i and COCO-20i demonstrate the superiority of DCP over conventional prototype-based approaches (up to 5~10% on average), establishing a new state-of-the-art.
Introduction
Few-shot segmentation is a challenging task that requires models to generalize to new categories using minimal supervision. The dominant prototype learning paradigm typically compresses foreground features from a support image into a single vector. However, this "squeezing" process is inherently lossy, often failing to capture the complexity of object boundaries or the presence of similar-looking distractors in the background.
As shown in Figure 1, conventional methods often result in incomplete segmentation or confusion with background elements. We address these issues by moving beyond a single prototype to a set of specialized "proxies" that explicitly model different aspects of the scene.
Methodology
1. The "Divide" Phase: Proxy Acquisition
Instead of relying solely on the provided ground-truth mask, DCP employs a self-reasoning scheme. The model first attempts to segment the support image itself. By comparing this prediction with the ground truth, the support image is decomposed into four distinct regions:
- $\alpha$ (Main Foreground): Regions correctly identified as foreground.
- $\beta$ (Boundary/Hard Foreground): Foreground regions missed by the model (e.g., thin parts or edges).
- $\gamma$ (Primary Background): Regions correctly identified as background.
- $\delta$ (Distractors): Background regions mistakenly identified as foreground.
Feature vectors, or proxies, are then extracted from these regions using masked average pooling, providing a multi-faceted representation of the target class and its environment.
2. The "Conquer" Phase: Parallel Decoder Structure (PDS)
To effectively utilize these proxies, we introduce a Parallel Decoder Structure. Simply combining all proxies into a single stream can lead to information confusion. Instead, our PDS uses two parallel branches:
- Foreground Information Flow: Utilizes $\alpha$ and $\beta$ proxies to refine object shape and boundaries.
- Background Information Flow: Utilizes $\gamma$ and $\delta$ proxies to suppress distractors and clarify the background.
Experimental Results
DCP was evaluated on two major benchmarks, PASCAL-5i and COCO-20i, where it established new state-of-the-art results. On PASCAL-5i, the model achieved a mean Intersection over Union (mIoU) of 61.31% in the 1-shot setting and 65.84% in the 5-shot setting using a VGG16 backbone. On the more complex COCO-20i dataset, DCP outperformed the previous best model (PFENet) by 2.89% (1-shot) and 3.78% (5-shot).
Qualitative results indicate that DCP is particularly effective at ignoring "distractor" objects—such as a cat when the target is a dog—and producing sharper, more accurate boundaries for complex shapes. The model maintains high efficiency, operating at over 16 FPS.
Conclusion
The Divide-and-Conquer Proxies (DCP) framework offers a robust alternative to standard prototype learning for few-shot segmentation. By leveraging the model's own reasoning failures to identify hard foreground and distractor regions, DCP provides superior guidance for segmenting unseen objects in complex scenes. Future work may extend this logic to related tasks like video object segmentation.
Summary updated