Weakly Supervised Semantic Segmentation via Progressive Confidence Region Expansion

Xiangfeng Xu, Pinyi Zhang, Wenxuan Huang, Yunhang Shen, Haosheng Chen, Jingzhong Lin, Wei Li, Gaoqi He, Jiao Xie, Shaohui LinCVPR 2025 · DOI

Introduction

Semantic segmentation is a fundamental computer vision task that involves assigning a class label to every pixel in an image. While high-performing models typically rely on dense, pixel-level annotations, the cost of such data is prohibitive. Weakly Supervised Semantic Segmentation (WSSS) addresses this by utilizing cheaper image-level labels. A common approach in WSSS is the use of Class Activation Maps (CAMs) to generate pseudo-labels. However, modern Vision Transformers (ViTs) often suffer from "over-expansion," where object heatmaps bleed into the background due to the global receptive field of self-attention. This paper introduces the Progressive Confidence Region Expansion (PCRE) framework to mitigate this issue.

Methodology

The PCRE framework is a one-stage approach designed to refine object boundaries by preventing the reckless expansion of activation regions. It consists of two primary components:

  1. Class-Prototype Enhancement (CPE): This module leverages dataset-wide class prototypes—global representations of object categories—to guide local feature learning. By using an Exponential Moving Average (EMA) to update these prototypes, the model maintains a stable "fingerprint" for each class, helping to distinguish object patches from background noise.
  2. Confidence Region Mask Expansion (CRME): Instead of thresholding a static heatmap, CRME starts with high-confidence seed regions and iteratively expands them. A patch is only added to the mask if it maintains a sufficient "Reference Overlap Ratio" (ROR) with the existing region, ensuring a controlled growth process.
The overall PCRE framework, featuring the Class-Prototype Enhancement (CPE) and Confidence Region Mask Expansion (CRME) modules for generating accurate pseudo-labels.

Experiments

The proposed method was evaluated on the PASCAL VOC 2012 and MS COCO 2014 benchmarks. PASCAL VOC contains 20 object classes, while MS COCO is a more complex dataset with 80 categories and diverse scenes. The primary metric used for evaluation is the mean Intersection over Union (mIoU).

Results

PCRE achieves state-of-the-art performance for single-stage WSSS. On PASCAL VOC, it reached 75.5% mIoU, surpassing previous methods like DIAL and DuPL. On the more challenging MS COCO dataset, it achieved 47.2% mIoU, representing a 2.8% improvement over existing benchmarks.

Qualitative analysis demonstrates that PCRE significantly reduces background "leakage." For instance, the over-activation rate for the "monitor" category was reduced by 27.8% compared to previous methods. As shown in Figure 1, PCRE produces much cleaner segmentation masks that respect object boundaries more effectively than standard ViT-based approaches.

Visual comparison of CAMs and segmentation results. PCRE (Ours) effectively suppresses background over-activation compared to baseline ViT and ToCo models.

Conclusion

The Progressive Confidence Region Expansion framework successfully addresses the over-expansion problem in ViT-based WSSS. By combining global class prototypes with an iterative, confidence-based mask expansion strategy, PCRE generates high-quality pseudo-labels that lead to superior segmentation performance. Future work may explore the application of this progressive logic to other forms of weak supervision, such as points or scribbles.

Summary updated

BibTeX
@inproceedings{xu2025weakly,
  author = {Xiangfeng Xu and Pinyi Zhang and Wenxuan Huang and Yunhang Shen and Haosheng Chen and Jingzhong Lin and Wei Li and Gaoqi He and Jiao Xie and Shaohui Lin},
  title = {{Weakly Supervised Semantic Segmentation via Progressive Confidence Region Expansion}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.00918},
}
Figure