Introduction
Semantic segmentation—the task of assigning a category label to every pixel in an image—is a cornerstone of computer vision but remains labor-intensive due to the need for precise, hand-drawn masks. Weakly-Supervised Semantic Segmentation (WSSS) aims to mitigate this by learning from image-level tags (e.g., "cat" or "bus").
The standard approach for WSSS utilizes Class Activation Maps (CAMs). However, CAMs typically suffer from "tunnel vision": a classifier only needs to identify the most discriminative part of an object (like a bird's beak) to classify the image correctly. This results in sparse attention maps that fail to cover the entire object, providing a poor foundation for full-object segmentation.
Methodology
This paper introduces a two-stage framework to overcome the limitations of sparse CAMs by leveraging the temporal dynamics of the training process.
Online Attention Accumulation (OAA)
The authors observed that while a network's attention is sparse at any single moment, its focus shifts across different training epochs. In one epoch, it might focus on a dog's face; in another, its paws. OAA maintains a cumulative attention map $M$ for every image. At each epoch $t$, the new attention map $A_t$ is merged with the previous cumulative map $M_{t-1}$ using an element-wise maximum operation: $$M_t = \max(M_{t-1}, A_t)$$ This allows the network to progressively "paint in" the entire object. To ensure quality, updates only occur when the classification confidence for the target category is high.
Integral Attention Learning (OAA+)
While accumulated maps are more complete, they may still contain low-confidence regions. The OAA+ stage treats these cumulative maps as soft labels to train a dedicated "Integral Attention Model." This model is optimized using a Hybrid Loss:
- Enhanced Loss: Encourages higher predictions for object regions, even those with low initial confidence.
- Constraint Loss: Penalizes the network for expanding into known background areas, preventing over-segmentation.
Experimental Results
The proposed method was evaluated on the PASCAL VOC 2012 benchmark, using a ResNet-101 backbone.
Quantitative Performance
OAA+ achieved state-of-the-art results for image-level supervised segmentation, reaching a 66.4% mIoU on the test set. This significantly outperforms previous leading methods such as SeeNet (60.7%) and AE-PSL (55.7%). Notably, the model is highly data-efficient; when trained on only 20% of the dataset, OAA+ still achieved 54.6% mIoU, surpassing several older methods trained on the full dataset.
Qualitative Analysis
Visual comparisons demonstrate that OAA and OAA+ produce significantly more complete and "thicker" object masks than standard CAMs. By accumulating shifting focal points, the model successfully captures the full extent of objects across diverse categories.
Conclusion
Integral Object Mining via OAA offers a simple, "plug-and-play" strategy for improving WSSS. By treating training as a temporal process, the method harvests far more information from standard classifiers than static approaches. While the reliance on external saliency maps for background cues remains a limitation, OAA provides a robust path toward more efficient and accurate weakly-supervised segmentation.
Summary updated