Anchor-Intermediate Detector: Decoupling and Coupling Bounding Boxes for Accurate Object Detection

Yilong Lv, Min Li, Yujie He, Zhuzhen He, Shaopeng Li, Aitao YangICCV 2023 · DOI · arXiv

Introduction

Object detection has traditionally been divided into two paradigms: anchor-based and anchor-free keypoint-based detectors. Anchor-based methods, such as RetinaNet, provide robust object identification but often lack precision in boundary localization. Conversely, anchor-free keypoint detectors like CornerNet offer high boundary precision but struggle with the "pairing problem"—the difficulty of correctly matching top-left and bottom-right corners, which often results in numerous false positives.

The Anchor-Intermediate Detector (AID) bridges this gap by decoupling predicted boxes into individual corners and re-coupling them based on localization confidence. This approach treats overlapping boxes not as redundant noise to be discarded by Non-Maximum Suppression (NMS), but as a valuable pool of candidate corners for refinement.

Methods and Architecture

The Anchor-Intermediate Detector (AID) is designed as a modular enhancement that can be integrated into various detector backbones. It employs a dual-head architecture that splits after a shared feature extraction stage (typically a ResNet or ResNeXt backbone):

  1. Anchor-based Head: This branch functions like a standard RetinaNet head, responsible for predicting object categories and initial bounding box coordinates relative to predefined anchors.
  2. Corner-aware Head: Operating in parallel, this anchor-free head generates heatmaps that predict the probability of each pixel being a top-left or bottom-right corner.

A key finding in the research was that training these heads in parallel—rather than sequentially—yielded a 1.3 AP improvement. This suggests that the features required for boundary perception are distinct from those used for general object classification and should be allowed to develop independently.

Box Decouple-Couple (BDC) Strategy

The defining feature of AID is the BDC strategy, which replaces or augments traditional NMS during the inference phase.

The BDC strategy refines object boundaries by aggregating corner points from multiple overlapping candidate boxes.

As illustrated in the figure above, the process involves three steps:

  • Decoupling: The top-ranked boxes from the anchor-based head are broken down into their constituent corners.
  • Scoring: Each corner is assigned a localization confidence score based on the heatmaps from the corner-aware head.
  • Coupling: The system re-pairs the highest-scoring top-left and bottom-right corners to synthesize a refined bounding box. This allows the model to leverage the best boundary information across multiple overlapping predictions.

Scoring and Localization Precision

To rank the final refined boxes, AID introduces the Corner-Classification (CoCl) score. This metric combines the semantic confidence of the anchor-based head with the geometric precision of the corner-aware head by multiplying the classification score by the corner localization confidence.

Scoring individual corners (tl and br) allows the model to select the most precise boundaries regardless of the initial anchor's classification score.

This scoring mechanism is particularly effective in complex scenes. As shown in the bus example, a box might have a high classification score but poor boundary alignment. By evaluating corners individually, AID can select the most precise boundaries (e.g., the blue box's top-left and the yellow box's bottom-right) to create a superior final detection. This strategy effectively sharpens object boundaries, leading to significant gains in high-precision metrics like $AP_{75}$.

Experimental Evaluation

The authors validated AID using the MS COCO dataset, specifically the train2017, val2017, and test-dev subsets. The results demonstrate that AID consistently improves localization accuracy across different baselines:

  • RetinaNet Baseline: Adding AID improved the Average Precision (AP) by 2.4 points on the COCO test-dev set.
  • GFL (Generalized Focal Loss) Baseline: Improved AP by 1.2 points.
  • State-of-the-Art Comparison: Using a ResNeXt-101 backbone, AID achieved 49.4 AP, outperforming several complex two-stage and anchor-free detectors without needing "bells and whistles" like multi-scale testing.

Conclusion

AID successfully mitigates the limitations of both anchor-based and keypoint-based detectors. By treating overlapping boxes as a source of boundary information rather than redundancy, the BDC strategy provides a robust mechanism for high-precision object localization. Future work will focus on reducing the training overhead through knowledge distillation and automating hyperparameter selection to make the model more efficient for real-time applications.

Summary updated

BibTeX
@inproceedings{lv2023anchor,
  author = {Yilong Lv and Min Li and Yujie He and Zhuzhen He and Shaopeng Li and Aitao Yang},
  title = {{Anchor-Intermediate Detector: Decoupling and Coupling Bounding Boxes for Accurate Object Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00577},
}
Figure