Abstract
Weakly supervised object localization (WSOL) aims to identify both class labels and locations of objects using only image-level labels. Most previous methods utilize the activation map corresponding to the highest activation source. Exploiting only one activation map of the highest probability class is often biased into limited regions or sometimes even highlights background regions. To resolve these limitations, we propose to use activation maps, named combinational class activation maps (CCAM), which are linear combinations of activation maps from the highest to the lowest probability class. By using CCAM for localization, we suppress background regions to help highlight foreground objects more accurately. In addition, we design the network architecture to consider spatial relationships for localizing relevant object regions. Specifically, we integrate non-local modules into an existing base network at both low- and high-level layers. Our final model, named non-local combinational class activation maps (NL-CCAM), obtains superior performance compared to previous methods on representative object localization benchmarks including ILSVRC 2016 and CUB-200-2011.
Introduction
Training object detectors typically requires expensive, manually drawn bounding boxes. Weakly Supervised Object Localization (WSOL) attempts to bypass this by using only image-level labels (e.g., "this is a picture of a bird") to determine exactly where the object is located.
The standard baseline for WSOL is the Class Activation Map (CAM), which identifies the spatial features that triggered the highest-probability class prediction. However, CAM suffers from two major flaws:
- The "Lazy Network" Problem: Classification networks only need to find the most discriminative part of an object to classify it (e.g., a bird's beak), often ignoring the rest of the body.
- Background Bleed: CAM often highlights common background regions associated with a class (e.g., highlighting water when trying to localize a ship).
Methodology
The proposed approach addresses these issues through two primary mechanisms: Combinational Class Activation Maps (CCAM) and Non-Local (NL) modules.
Combinational Class Activation Maps (CCAM)
Instead of relying solely on the Top-1 predicted class map, CCAM extracts maps for classes ranked from highest to lowest probability. A combination function adds the top maps and subtracts the bottom maps. This subtraction actively suppresses background noise, as low-probability classes naturally highlight background regions that the network has learned to distinguish from the target classes.
Non-Local (NL) Modules
To encourage the network to localize the entire object rather than just discriminative parts, non-local attention blocks are inserted into both low-level and high-level layers of a VGG16 backbone. These modules capture long-range spatial dependencies, allowing the network to associate distant parts of an object, such as linking a bird's tail feathers to its beak.
Experimental Results
The models were trained and evaluated on two standard WSOL benchmarks: ILSVRC 2016 (ImageNet) and CUB-200-2011.
- State-of-the-Art Localization: On ILSVRC 2016, NL-CCAM achieved a Top-1 localization error of 49.83%, outperforming the previous state-of-the-art, SPG (51.40%). On CUB-200-2011, it hit 47.60% error (vs. SPG's 53.36%).
- Improved Classification: Unlike erasing-based WSOL methods that often degrade classification accuracy, adding non-local modules improved Top-1 classification error on ILSVRC to 27.7% (compared to the VGGnet-GAP baseline of 33.4%).
- Ablation Success: Using CCAM alone (without the non-local modules) still improved Top-1 localization on CUB by ~7.9% over the baseline, proving the background-suppression concept works independently of the attention mechanism.
Discussion and Conclusion
A key insight of this research is that low-probability classes serve as excellent background detectors. Because the network minimizes the score of incorrect classes, it isolates non-discriminative background regions in those specific maps. CCAM provides background suppression with minimal computational overhead, as it reuses activation maps already generated during the standard forward pass.
While the choice of the combination function is currently dataset-dependent (requiring manual tuning for diverse vs. fine-grained datasets), the overall framework demonstrates that leveraging the full spectrum of class activations significantly advances the state of weakly supervised object localization. Future work may focus on learning these combination heuristics dynamically during training.
Summary updated