Learning Object Scale With Click Supervision for Object Detection

Liao Zhang, Yan Yan, Lin Cheng, Hanzi Wang

TL;DR

This paper introduces a novel weakly-supervised object detection method that leverages single-click annotations to accurately estimate object scale. By combining this click supervision with a new CNN visualization technique called Spatial Attention CAM (SA-CAM) and a proposal selection strategy, the method generates high-quality pseudo ground-truths for training a fully-supervised detector, achieving state-of-the-art performance.

What Problem Does This Paper Address?

Object detection, a fundamental task in computer vision, typically relies on expensive bounding box annotations for training. While weakly-supervised methods that use only image-level labels reduce annotation costs, they often suffer from significantly lower performance. This paper addresses the challenge of achieving a good balance between annotation effort and detection accuracy by exploring the use of minimal supervision. Specifically, it tackles the problem of accurately estimating object scale, which is crucial for robust detection, especially for objects of varying sizes, using a more efficient annotation scheme than full bounding boxes.

Methods

The proposed method combines several key components:

  • Center Click Supervision: Instead of full bounding boxes, the method uses a single click at the object's center as the primary annotation. This significantly reduces annotation time while providing a strong anchor for object localization.
  • Proposal Selection: To improve the quality of training data, the authors employ a selective search algorithm to generate object proposals. These proposals are then filtered based on the presence of center clicks and ranked by their proximity to the click. Highly overlapping proposals are further removed to ensure diversity and focus on relevant regions.
  • Spatial Attention CAM (SA-CAM): This is the core novel contribution for generating pseudo ground-truths. Building upon Grad-CAM, SA-CAM refines the process of visualizing class-specific regions within CNN feature maps. It incorporates a ReLU function to preserve only positive gradients and uses pixel-wise multiplication with gradients to assign attention scores, leading to more precise localization of object scales.
  • Pseudo Ground-Truth Generation: The SA-CAMs are generated for selected proposals, upsampled to their original sizes, projected back onto the image, and fused to create a high-resolution heatmap. This heatmap, combined with the center click information, is used to generate accurate pseudo bounding boxes.
  • Fully-Supervised Detector Training: The generated pseudo ground-truths are then used to train a standard Fast-RCNN detector, leveraging the power of fully-supervised learning with reduced annotation effort.

Datasets

  • PASCAL VOC 2007: Used for both training and evaluation.
  • PASCAL VOC 2012: Used for evaluation.
  • Center Click Annotation Data: The authors utilize center click annotations provided by a previous work [17].

Key Insights and Ideas

  • The Power of Minimal Supervision: A single click at the object's center is surprisingly effective for guiding object scale estimation, significantly outperforming image-level supervision alone.
  • SA-CAM's Superiority: The proposed SA-CAM method offers a more accurate way to visualize discriminative object regions compared to existing methods like Grad-CAM, leading to better pseudo ground-truth generation.
  • Proposal Selection is Crucial: Filtering and ranking object proposals before training the classification network helps to reduce background noise and focus on relevant object parts, improving scale estimation.
  • Trade-off in Gradient Back-propagation: Back-propagating gradients from intermediate convolutional layers (like conv5_2) provides a better balance of semantic information and spatial detail for object localization compared to very early or very late layers.

Key Results

  • PASCAL VOC 2007: The method achieved 78.2% CorLoc and 56.8% mAP using VGG16 as the backbone, significantly outperforming state-of-the-art image-level based methods and a center-click based method. When compared with the center-click method using AlexNet, the proposed method achieved 47.6% mAP versus 45.9% mAP.
  • PASCAL VOC 2012: The method achieved 78.2% CorLoc and 54.6% mAP, again outperforming existing methods.
  • Qualitative Results: Examples show that the pseudo ground-truths generated by the proposed method are tighter and more accurately cover object regions compared to the center-click method.
  • SA-CAM vs. Grad-CAM: SA-CAM achieved 79.6% CorLoc, outperforming Grad-CAM (74.0% CorLoc) while maintaining a comparable running speed (8.6 fps vs. 8.4 fps).

Limitations and Future Work

  • Limitations: The paper acknowledges that while the proposed method significantly improves performance, it still lags behind fully-supervised methods. The accuracy of pseudo ground-truths can be affected by challenging samples with similar backgrounds or difficult instances.
  • Future Work: The authors suggest that further improvements could be achieved by exploring more advanced CNN visualization techniques and more sophisticated proposal selection strategies. The paper also implicitly points towards the potential of using even less supervision or exploring different annotation modalities.

Summary updated

Abstract

Weakly-supervised object detection has recently attracted increasing attention since it only requires image-levelannotations. However, the performance obtained by existingmethods is still far from being satisfactory compared with fully-supervised object detection methods. To achieve a good trade-off between annotation cost and object detection performance,we propose a simple yet effective method which incorporatesCNN visualization with click supervision to generate the pseudoground-truths (i.e., bounding boxes). These pseudo ground-truthscan be used to train a fully-supervised detector. To estimatethe object scale, we firstly adopt a proposal selection algorithmto preserve high-quality proposals, and then generate ClassActivation Maps (CAMs) for these preserved proposals by theproposed CNN visualization algorithm called Spatial AttentionCAM. Finally, we fuse these CAMs together to generate pseudoground-truths and train a fully-supervised object detector withthese ground-truths. Experimental results on the PASCAL VOC2007 and VOC 2012 datasets show that the proposed methodcan obtain much higher accuracy for estimating the object scale,compared with the state-of-the-art image-level based methodsand the center-click based method
BibTeX
@article{zhang2019learning,
  author = {Liao Zhang and Yan Yan and Lin Cheng and Hanzi Wang},
  title = {{Learning Object Scale With Click Supervision for Object Detection}},
  journal = {IEEE Signal Processing Letters},
  year = {2019},
  doi = {10.1109/lsp.2019.2937387},
}
Figure