A Consistency and Integration Model with Adaptive Thresholds for Weakly Supervised Object Localization

Hao Su, Meng YangIJCAI 2024

Abstract

Weakly Supervised Object Localization (WSOL) aims to identify object boundaries using only image-level labels, offering a cost-effective alternative to fully supervised detection. However, standard models often fall into the "discriminative trap," focusing solely on the most salient object parts, and struggle with fixed thresholding for bounding box generation. We present the Consistency and Integration Model with Adaptive Thresholds (CIAT), which addresses these issues through two novel components: a Consistency and Integration of Shallow-Deep features (CISD) module and an Online Adaptive Threshold (OAT) mechanism. By leveraging the spatial richness of shallow layers and the statistical robustness of Bayesian decision theory, CIAT achieves state-of-the-art localization accuracy on the ILSVRC and CUB-200-2011 benchmarks.

Introduction

Weakly Supervised Object Localization (WSOL) is a critical task in computer vision that seeks to locate objects using only category-level annotations. While significantly reducing the cost of data labeling, WSOL faces two primary challenges. First, the Discriminative Trap: classification-trained models naturally gravitate toward the most distinctive features (e.g., a bird's beak) rather than the entire object body. Second, the Threshold Problem: the use of a fixed global threshold to convert heatmaps into bounding boxes fails to account for varying image conditions, leading to inconsistent localization.

Methodology

The CIAT framework introduces a dual-pronged approach to bridge the gap between classification and localization.

Consistency and Integration of Shallow-Deep features (CISD)

While deep layers in Vision Transformers (ViT) capture high-level semantics, shallow layers preserve crucial spatial details like edges and textures. The CISD module calculates the consistency between these layers to generate a "spatial-awareness class attention" map. This integration ensures the model maintains a holistic view of the object, preventing it from collapsing onto small, highly discriminative regions.

Online Adaptive Threshold (OAT)

To move beyond brittle fixed thresholds, CIAT employs the Online Adaptive Threshold (OAT) module. OAT re-frames localization as a binary classification problem between foreground and background pixels. By modeling pixel intensities as Gaussian distributions and applying Bayesian decision theory, OAT calculates an optimal, image-specific threshold that minimizes classification error. This makes the resulting bounding boxes significantly more robust to variations in lighting and background clutter.

The CIAT architecture featuring the Consistency and Integration of Shallow-Deep features (CISD) module and the Online Adaptive Threshold (OAT) process.

Experimental Results

CIAT was evaluated on two major benchmarks: CUB-200-2011 (fine-grained bird species) and ILSVRC (large-scale ImageNet).

Quantitative Performance

CIAT established new state-of-the-art (SOTA) benchmarks:

  • ILSVRC: Achieved 59.8% Top-1 Localization accuracy, a 2.9% improvement over the previous best method (CATR).
  • CUB-200-2011: Reached 97.1% GT-known localization accuracy, demonstrating performance nearly on par with fully supervised models.

Qualitative Analysis

Visualizations confirm that CIAT produces more complete bounding boxes compared to previous methods like TS-CAM and SCM. While baselines often miss extremities (e.g., tails or ears), CIAT successfully integrates these regions into the final localization map, leading to much more "human-like" bounding boxes.

Qualitative localization results on CUB (left) and ILSVRC (right) benchmarks, comparing CIAT with baseline methods. Numbers indicate localization scores.

Conclusion and Future Work

CIAT demonstrates that integrating shallow spatial features and employing adaptive statistical thresholding can significantly overcome the inherent limitations of weakly supervised localization. While the current model assumes Gaussian distributions for pixel intensities—a potential limitation in highly complex scenes—it provides a robust "plug-and-play" solution for modern Vision Transformers. Future work may involve integrating the Bayesian thresholding logic directly into the training objective to further refine the model's spatial perception.

Summary updated

BibTeX
@inproceedings{su2024consistency,
  author = {Hao Su and Meng Yang},
  title = {{A Consistency and Integration Model with Adaptive Thresholds for Weakly Supervised Object Localization}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure