Square Localization for Efficient and Accurate Object Detection

Cewu Lu, Yongyi Lu, Hao Chen, Chi-Keung TangICCV 2015 · OpenReview

Abstract

The key contribution of this paper is the compact square object localization, which relaxes the exhaustive sliding window from testing all windows of different combinations of aspect ratios. Square object localization is category scalable. By using a binary search strategy, the number of scales to test is further reduced empirically to only O(log(min{H, W})) rounds of sliding CNNs, where H and W are respectively the image height and width. In the training phase, square CNN models and object co-presence priors are learned. In the testing phase, sliding CNN models are applied which produces a set of response maps that can be effectively filtered by the learned co-presence prior to output the final bounding boxes for localizing an object. We performed extensive experimental evaluation on the VOC 2007 and 2012 datasets to demonstrate that while efficient, square localization can output precise bounding boxes to improve the final detection result.

Introduction

Object detection traditionally relies on the exhaustive sliding window approach, where an algorithm scans an image across all possible positions, scales, and aspect ratios. This results in a search space complexity of $O(HW \cdot \text{scales} \cdot \text{aspect ratios})$, which is computationally prohibitive. To mitigate this, modern detectors often rely on external object proposal methods like Selective Search or Edge Boxes. This paper proposes a paradigm shift: Square Localization. By focusing exclusively on "Compact Square Objects" (CSOs), the method eliminates the need to search across multiple aspect ratios, significantly streamlining the detection pipeline without sacrificing semantic recognition capability.

Methodology

The proposed framework simplifies object localization into a search for square regions that represent the core of an object.

Compact Square Objects (CSOs)

A CSO is defined as the largest square that can be inscribed within an object's tight bounding box. The authors hypothesize that these square regions contain sufficient semantic information for accurate classification. Validation tests showed that humans can identify objects from these square crops with 95.4% accuracy, nearly matching the 97.6% accuracy achieved on full images. This confirms that the aspect ratio is largely redundant for the initial recognition and localization phase.

Square crops (CSOs) used for training and recognition.

Efficient Search via Binary Scaling

By restricting the search to squares, the aspect ratio variable is removed. To further optimize the scale search, the authors implement a recursive binary search strategy. Instead of testing every possible size, the algorithm compares response maps at different scales; if the maps are sufficiently similar, intermediate scales are skipped. This reduces scale complexity to $O(\log(\min\{H, W\}))$.

Sliding square windows at varying positions and scales.

Object Co-presence Prior

To handle the clutter generated by sliding windows, the authors introduce a statistical "co-presence prior" learned from the ILSVRC dataset. This prior models the spatial distribution of indicator pixels, effectively filtering out noise and redundant detections in a manner superior to standard Non-Maximum Suppression (NMS). Finally, the algorithm concatenates the detected square regions to reconstruct the original rectangular bounding box.

Experimental Results

The method was evaluated on the PASCAL VOC 2007 and 2012 benchmarks. On VOC 2007, Square Localization achieved a Mean Average Precision (mAP) of 62.0%, notably outperforming the R-CNN baseline (58.5%) which utilizes Selective Search.

Furthermore, the method demonstrates exceptional localization efficiency, reaching an 82% localization rate with only 100 bounding box proposals per image. This efficiency is attributed to the logarithmic growth of CNN passes relative to image size and the category scalability of the model, where the first six layers of the CNN are shared across all object classes.

Conclusion

Square Localization demonstrates that searching for aspect ratios is largely redundant for object localization. By focusing on the square "core" of objects and utilizing binary search for scale, the proposed method achieves high accuracy with significantly reduced computational complexity. While the co-presence prior handles most clutter, future work may address extreme proximity cases where objects of the same category overlap significantly.

Summary updated

Abstract

The key contribution of this paper is the compact square object localization, which relaxes the exhaustive sliding window from testing all windows of different combinations of aspect ratios. Square object localization is category scalable. By using a binary search strategy, the number of scales to test is further reduced empirically to only O(log(min{H, W})) rounds of sliding CNNs, where H and W are respectively the image height and width. In the training phase, square CNN models and object co-presence priors are learned. In the testing phase, sliding CNN models are applied which produces a set of response maps that can be effectively filtered by the learned co-presence prior to output the final bounding boxes for localizing an object. We performed extensive experimental evaluation on the VOC 2007 and 2012 datasets to demonstrate that while efficient, square localization can output precise bounding boxes to improve the final detection result.
BibTeX
@inproceedings{lu2015square,
  author = {Cewu Lu and Yongyi Lu and Hao Chen and Chi-Keung Tang},
  title = {{Square Localization for Efficient and Accurate Object Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2015},
  url = {https://openreview.net/forum?id=HJEEZ-fuWS},
}
Figure