Introduction
Semantic segmentation is a fundamental computer vision task that involves assigning a class label to every pixel in an image. While high-performing models typically rely on dense, pixel-level annotations, the cost of such data is prohibitive. Weakly Supervised Semantic Segmentation (WSSS) addresses this by utilizing cheaper image-level labels. A common approach in WSSS is the use of Class Activation Maps (CAMs) to generate pseudo-labels. However, modern Vision Transformers (ViTs) often suffer from "over-expansion," where object heatmaps bleed into the background due to the global receptive field of self-attention. This paper introduces the Progressive Confidence Region Expansion (PCRE) framework to mitigate this issue.
Methodology
The PCRE framework is a one-stage approach designed to refine object boundaries by preventing the reckless expansion of activation regions. It consists of two primary components:
- Class-Prototype Enhancement (CPE): This module leverages dataset-wide class prototypes—global representations of object categories—to guide local feature learning. By using an Exponential Moving Average (EMA) to update these prototypes, the model maintains a stable "fingerprint" for each class, helping to distinguish object patches from background noise.
- Confidence Region Mask Expansion (CRME): Instead of thresholding a static heatmap, CRME starts with high-confidence seed regions and iteratively expands them. A patch is only added to the mask if it maintains a sufficient "Reference Overlap Ratio" (ROR) with the existing region, ensuring a controlled growth process.
Experiments
The proposed method was evaluated on the PASCAL VOC 2012 and MS COCO 2014 benchmarks. PASCAL VOC contains 20 object classes, while MS COCO is a more complex dataset with 80 categories and diverse scenes. The primary metric used for evaluation is the mean Intersection over Union (mIoU).
Results
PCRE achieves state-of-the-art performance for single-stage WSSS. On PASCAL VOC, it reached 75.5% mIoU, surpassing previous methods like DIAL and DuPL. On the more challenging MS COCO dataset, it achieved 47.2% mIoU, representing a 2.8% improvement over existing benchmarks.
Qualitative analysis demonstrates that PCRE significantly reduces background "leakage." For instance, the over-activation rate for the "monitor" category was reduced by 27.8% compared to previous methods. As shown in Figure 1, PCRE produces much cleaner segmentation masks that respect object boundaries more effectively than standard ViT-based approaches.
Conclusion
The Progressive Confidence Region Expansion framework successfully addresses the over-expansion problem in ViT-based WSSS. By combining global class prototypes with an iterative, confidence-based mask expansion strategy, PCRE generates high-quality pseudo-labels that lead to superior segmentation performance. Future work may explore the application of this progressive logic to other forms of weak supervision, such as points or scribbles.
Summary updated