Introduction
Weakly Supervised Object Localization (WSOL) aims to localize objects within an image using only image-level category labels during training. This task is challenging because the model must infer spatial extents without explicit bounding box supervision. Traditional methods relying on Class Activation Maps (CAMs) often focus on the most discriminative parts of an object (e.g., a bird's head) rather than its full extent. While Vision Transformers (ViTs) offer long-range dependencies that help capture global context, their self-attention maps are typically category-agnostic, leading to noisy localization that fails to distinguish between different classes in a scene.
Methodology: The CATR Framework
The proposed Category-aware Allocation Transformer (CATR) addresses these limitations by forcing the transformer's internal attention mechanisms to become class-specific. The architecture introduces two primary components: the Category-aware Stimulation Module (CSM) and the Object Constraint Module (OCM).
Category-aware Stimulation Module (CSM)
The CSM is designed to "stimulate" the transformer's attention maps to learn category-specific features. It takes raw attention maps and processes them through a CNN-based encoder-decoder structure. By applying an auxiliary classification loss ($L_{csm}$) to these maps, the model is incentivized to highlight regions that are not just "objects" but are specifically relevant to the target class.
Object Constraint Module (OCM)
The OCM refines the localization by addressing two common issues: background noise and incomplete object coverage.
- Noise Suppression: A percentile-based thresholding technique is used to filter out low-activation regions, effectively silencing background clutter.
- Object Awakening: This mechanism aggregates features across multiple transformer blocks to generate pixel-level pseudo-labels. These labels provide a self-supervised signal that helps the model "wake up" the full geometry of the object, ensuring that non-discriminative but relevant parts (like a bird's tail) are included in the final localization map.
To ensure stable training, CATR employs a learnable weighting mechanism that automatically balances the primary classification loss with the auxiliary losses from the CSM and OCM.
Experiments and Results
The CATR model was evaluated on two major benchmarks: CUB-200-2011 (fine-grained bird species) and ILSVRC (ImageNet).
Quantitative Performance
CATR achieved state-of-the-art results across both datasets:
- CUB-200-2011: The model reached a 79.62% Top-1 Localization Accuracy, significantly outperforming previous transformer-based methods like LCTR.
- ILSVRC: CATR achieved 56.90% Top-1 Localization Accuracy, a substantial improvement over the TS-CAM baseline (53.40%).
Qualitative Analysis
Visualizations demonstrate that CATR produces more comprehensive localization maps compared to previous methods. While standard CAMs often highlight only small, highly discriminative regions, CATR's category-aware attention maps consistently cover the entire body of the target object with tighter boundaries and less background interference.
Conclusion
By integrating category awareness into the transformer's self-attention mechanism, CATR effectively bridges the gap between global feature dependency and class-specific localization. The combination of the Category-aware Stimulation Module and the Object Constraint Module allows the model to suppress noise and capture the full extent of objects, setting a new benchmark for weakly supervised object localization.
Summary updated