Introduction
Weakly Supervised Semantic Segmentation (WSSS) aims to train segmentation models using only image-level labels, significantly reducing the cost of pixel-level annotation. The primary challenge in WSSS is the "discriminative region" problem associated with Class Activation Maps (CAMs). CAMs often only highlight the most salient parts of an object (e.g., a bird's head) while ignoring the rest of the body or including background noise. While contrastive learning has been applied to address this, most existing methods focus on a single semantic level, failing to capture the full structural context of a scene.
Methodology: Hierarchical Semantic Contrast (HSC)
This paper introduces Hierarchical Semantic Contrast (HSC), a framework that learns object patterns across three distinct semantic levels: Region of Interest (ROI), class, and pixel. By supervising these levels simultaneously, the model develops a more robust understanding of "objectness."
1. ROI-level Contrast
The ROI-level contrast distinguishes the general foreground from the background. This prevents the embedding space from collapsing and helps the model localize where objects generally exist within the scene.
2. Class-level Contrast
To ensure consistency across different images, the framework utilizes Momentum Prototypes. These prototypes act as a global memory, storing running averages of class features across the dataset. This ensures that features for a specific class (e.g., "cat") remain similar across different instances and distinct from other classes.
3. Pixel-level Contrast
Pixel-level contrast refines fine-grained details. To mitigate the noise inherent in weak labels, the authors employ a Conditional Random Field (CRF) to refine pseudo-labels before using them for contrastive supervision.
The architecture is built upon a Siamese Network where two augmented views of the same image are processed. Cross-view supervision is applied, where the hierarchical semantics from one view serve as the ground truth for the other, enforcing spatial and scale consistency.
Experimental Results
The HSC framework was evaluated on the PASCAL VOC 2012 benchmark. It achieved a new state-of-the-art performance with 73.6% mIoU on the validation set and 74.5% on the test set. Notably, when integrated into existing baselines like SEAM, HSC provided an 8.9% mIoU boost to the initial seed quality. Qualitative results demonstrate that HSC produces more complete masks with sharper boundaries, particularly in challenging categories like "plant" and "sofa."
Conclusion
Hierarchical Semantic Contrast offers a comprehensive approach to WSSS by leveraging multi-level semantic structures. By combining ROI, class, and pixel-level supervision within a siamese framework, HSC effectively addresses the limitations of standard CAM-based methods. Future work may involve integrating other forms of weak supervision, such as points or scribbles, to further narrow the gap with fully supervised methods.
Summary updated