Abstract Weakly supervised semantic segmentation (WSSS) using image-level classification labels usually utilizes the Class Activation Maps (CAMs) to localize objects of interest in images. While pointing out that CAMs only highlight the most discriminative regions of the classes of interest, adversarial erasing (AE) methods have been proposed to further explore the less discriminative regions. In this paper, we review the potential of the pre-trained classifier which is trained on the raw images. We experimentally verify that the ordinary classifier already has the capability to activate the less discriminative regions if the most discriminative regions are erased to some extent. Based on that, we propose a class-specific AE-based framework that fully exploits the potential of an ordinary classifier. Our framework (1) adopts the ordinary classifier to notify the regions to be erased and (2) generates a class-specific mask for erasing by randomly sampling a single specific class to be erased (target class) among the existing classes on the image for obtaining more precise CAMs. Specifically, with the guidance of the ordinary classifier, the proposed CAMs Generation Network (CGNet) is enforced to generate a CAM of the target class while constraining the CAM not to intrude the object regions of the other classes. Along with the pseudo-labels refined from our CAMs, we achieve the state-of-the-art WSSS performance on both PASCAL VOC 2012 and MS-COCO dataset only with image-level supervision.
1. Introduction
Semantic segmentation—the task of labeling every pixel in an image—usually requires massive amounts of expensive, hand-drawn pixel-level annotations. To avoid this cost, researchers use Weakly Supervised Semantic Segmentation (WSSS), which attempts to learn pixel-level details using only image-level tags (e.g., "this image contains a dog").
The standard tool for this is the Class Activation Map (CAM). However, CAMs suffer from a "sparsity" problem: a classifier trained to recognize a dog might only focus on the dog's face because that is the most discriminative part. Consequently, the resulting segmentation mask only covers the face, missing the rest of the body. Previous "Adversarial Erasing" (AE) methods attempted to fix this by hiding the most discriminative regions and training a new classifier to find the remainder, but this process is computationally heavy and often leads to imprecise object boundaries.
2. Methodology: Exploiting the Ordinary Classifier
The authors propose a framework that shifts the focus from training new classifiers to better utilizing an "Ordinary Classifier" (one pre-trained on raw images). Through a pilot experiment, the authors discovered that if a fixed, pre-trained classifier is presented with an image where the most active regions are masked, the same classifier naturally begins activating on the next most relevant parts of the object. This suggests that the classifier already possesses latent knowledge of the entire object; it simply does not report it during standard inference because the most discriminative features are sufficient for classification.
2.1 CGNet and Class-Specific Erasing (CSE)
The framework consists of the CAM Generation Network (CGNet), which is the model being trained, and a fixed Ordinary Classifier. Unlike previous class-agnostic methods that erased all foreground objects simultaneously, this method introduces Class-Specific Erasing (CSE). The system randomly selects one "target class" to erase from the image.
The masked image is then fed to the fixed Ordinary Classifier. The system utilizes a Class-Specific Erasing loss ($L_{cse}$) to guide the CGNet:
- It penalizes the CGNet if the Ordinary Classifier can still detect the target class in the masked image.
- Crucially, it penalizes the CGNet if the generated mask accidentally covers other classes present in the image. This "boundary awareness" prevents the mask from bleeding into neighboring objects.
3. Experimental Results
The researchers evaluated their framework on two industry-standard benchmarks: PASCAL VOC 2012 (20 foreground classes) and MS-COCO (81 classes). The latter is significantly more challenging due to its dense, multi-object scenes.
3.1 Performance Metrics
The framework achieved State-of-the-Art (SOTA) performance for WSSS using only image-level labels:
- PASCAL VOC 2012: Reached 68.4% mIoU (mean Intersection over Union) on the test set, outperforming previous leading methods such as BES and ICD.
- MS-COCO: Achieved 36.4% mIoU, representing a significant 3.0% improvement over the previous best method (IRNet+CONTA).
Qualitative analysis shows that the generated masks are more "complete" (covering the entire object) and "sharp" (matching actual edges) compared to baseline CAMs. This is attributed to the fixed guidance of the Ordinary Classifier, which provides a more stable gradient than "self-guidance" methods where the teacher and student models change simultaneously.
4. Discussion and Conclusion
This research reveals that standard image classifiers are "lazy" but knowledgeable. They focus on "low-hanging fruit" to minimize loss but retain latent knowledge of entire objects. By forcing the model to look closer through class-specific erasing, the proposed framework unlocks this potential without the need for additional pixel-level supervision.
A primary limitation remains the reliance on the Ordinary Classifier itself. If the pre-trained classifier misidentifies an object (e.g., mistaking a cat for a dog), the erasing process will be fundamentally flawed. Future work may focus on more robust training schemes to handle these misclassifications or exploring specialized classifiers designed to identify less discriminative regions from the outset.
Summary updated