Abstract
Weakly supervised object localization (WSOL) has recently attracted attention since it aims to identify both class labels and locations of objects by using image-level labels. Most previous methods utilize the activation map corresponding to the highest activation source. Exploiting only one activation map of the highest probability class is often biased into limited regions or sometimes even highlights background regions. To resolve these limitations, we propose to use activation maps, named combinational class activation maps (CCAM), which are linear combinations of activation maps from the highest to the lowest probability class. By using CCAM for localization, we suppress background regions to help highlighting foreground objects more accurately. In addition, we design the network architecture to consider spatial relationships for localizing relevant object regions. Specifically, we integrate non-local modules into an existing base network at both low- and high-level layers. Our final model, named non-local combinational class activation maps (NL-CCAM), obtains superior performance compared to previous methods on representative object localization benchmarks including ILSVRC 2016 and CUB-200-2011.
Introduction
Weakly Supervised Object Localization (WSOL) is a cost-effective alternative to standard object detection. Instead of requiring labor-intensive bounding box annotations, WSOL models are trained using only image-level labels (e.g., "bird," "car"). The standard baseline for this task is Class Activation Mapping (CAM), which identifies the pixels that most contribute to a classifier's prediction.
However, CAM suffers from two primary limitations: discriminative bias and background noise. Discriminative bias occurs when a model focuses only on the most distinct part of an object (like a bird's beak) rather than its full extent. Background noise arises when the model incorrectly activates on elements that frequently co-occur with the target object, such as water appearing with boats or grass with animals.
Methodology
Combinational Class Activation Maps (CCAM)
To address background noise, the authors introduce CCAM. While traditional CAM only utilizes the activation map for the highest-probability class, CCAM leverages the insight that low-probability class maps often contain useful information about what the network considers "background."
By creating a linear combination of maps from the highest to the lowest probability classes, the model can effectively "subtract" background regions. The researchers explored various weighting functions, including polynomial curves and a "Top-i & Bottom-j" approach, to find the optimal balance for suppressing non-object regions while preserving the foreground.
Non-Local Modules
To combat discriminative bias, the architecture integrates non-local modules into a VGG16 backbone. Unlike standard convolutions that operate locally, non-local modules compute relationships between all pixels in a feature map. By applying these modules at both low- and high-level layers, the network can capture long-range spatial dependencies. This allows the model to recognize that disparate parts of an object (e.g., a bird's head and tail) are spatially related, leading to more complete and accurate bounding boxes.
Experimental Results
The proposed NL-CCAM was evaluated on the ILSVRC 2016 (ImageNet) and CUB-200-2011 datasets, achieving state-of-the-art results. On ImageNet, it achieved a Top-1 localization error of 49.83%, a significant improvement over the original CAM (57.20%). On the fine-grained CUB-200-2011 dataset, it reduced the Top-1 localization error to 47.60%.
Qualitative analysis (as seen in the figure above) demonstrates that the combination of multiple class maps ($M^{c_1}$ through $M^{c_K}$) allows the model to suppress background clutter that often confuses standard CAM. For instance, in the lighthouse example, standard CAM activates on the surrounding trees and sky, whereas NL-CCAM tightly localizes the structure itself.
Conclusion
NL-CCAM demonstrates that the "wrong" class predictions in a classifier are not merely failures but valuable spatial indicators of the background. By combining these maps and utilizing non-local modules for better spatial context, the method provides a robust, single-forward-pass solution for weakly supervised localization. Future work may involve adaptive weighting mechanisms to further refine how these maps are combined for diverse image types.
Summary updated