Evolving Boxes for fast Vehicle Detection

Li Wang, Yao Lu, Hong Wang, Yingbin Zheng, Hao Ye, Xiangyang Xue

Abstract

We perform fast vehicle detection from traffic surveillance cameras. A novel deep learning framework, namely Evolving Boxes, is developed that proposes and refines the object boxes under different feature representations. Specifically, our framework is embedded with a light-weight proposal network to generate initial anchor boxes as well as to early discard unlikely regions; a fine-turning network produces detailed features for these candidate boxes. We show intriguingly that by applying different feature fusion techniques, the initial boxes can be refined for both localization and recognition. We evaluate our network on the recent DETRAC benchmark and obtain a significant improvement over the state-of-the-art Faster RCNN by 9.5% mAP. Further, our network achieves 9-13 FPS detection speed on a moderate commercial GPU.

Introduction

Vehicle detection in real-world traffic surveillance is notoriously difficult due to extreme variations in lighting, weather, vehicle pose, and frequent occlusions. While general-purpose detectors like Faster R-CNN have set high standards, they often struggle with the specific constraints of surveillance: the need to detect very small objects (vehicles far from the camera) and the requirement for high efficiency in processing dense video streams.

The authors identify a gap in how current models handle "candidate boxes." Instead of regressing a box once from a single set of features, they propose that boxes should "evolve"—being filtered and refined multiple times using different levels of visual information.

Methodology

The Evolving Boxes (EB) framework consists of three primary components integrated into a single end-to-end pipeline:

  1. Deep Convolutional Network (DCN): Acts as the backbone (based on VGG-16) to extract rich feature representations from the input image.
  2. Proposal Network (PN): A light-weight stage that generates initial "anchor boxes." Its primary job is efficiency; it quickly discards roughly 98% of background regions that clearly don't contain vehicles, passing only high-probability candidates to the next stage.
  3. Fine-Tuning Network (FTN): This stage performs the heavy lifting. It takes the candidates from the PN and applies more sophisticated convolutional layers. Crucially, it uses Feature Fusion, concatenating features from the PN and the FTN to produce the final "evolved" bounding box and classification score.
The Evolving Boxes (EB) architecture, featuring the DCN backbone, Proposal Network (PN), and Fine-Tuning Network (FTN).

The system is trained using a multi-stage loss function that balances the classification and localization errors of both the PN and FTN simultaneously.

Experimental Setup

The researchers evaluated their framework using the DETRAC (Detection and Tracking) benchmark, which includes 140K captured frames with 1.2M labeled vehicles. The data was captured at various locations in Beijing under diverse conditions: sunny, cloudy, rainy, and night. The dataset is particularly challenging due to high vehicle density (averaging 8.6 vehicles per frame) and frequent occlusions.

Key Results

The Evolving Boxes model demonstrates superior performance across accuracy and efficiency metrics:

  • Accuracy: On the DETRAC test set, the EB model achieved a 67.96% mAP, significantly outperforming Faster R-CNN (58.45%) and other models like CompACT (53.23%) and R-CNN (48.95%).
  • Sub-category Performance: EB performed best across all tested scenarios, including "Easy," "Medium," "Hard," and various weather conditions.
  • Ablation Gains: The inclusion of multi-layer feature fusion alone provided a massive 10.5% mAP boost, proving that spatial detail is critical in surveillance tasks.
  • Efficiency: The model runs at 9-13 FPS (approx. 110ms per frame) on a moderate commercial GPU (Nvidia Titan X Maxwell), making it viable for near-real-time applications.
Precision-Recall curves on the DETRAC benchmark, showing EB outperforming state-of-the-art detectors.

Discussion and Conclusion

The core insight of this work is that localization shouldn't be a one-shot process. By allowing the PN to provide a "rough draft" and the FTN to "edit" that draft using higher-resolution features, the final boxes are significantly more accurate. Furthermore, by using a light-weight PN to filter out the majority of the background, the model avoids wasting expensive computational resources on empty regions.

While the results are impressive, the authors noted specific failure cases such as "Box Splitting" (splitting a single large vehicle into multiple boxes) and difficulty distinguishing between adjacent or overlapping vehicles. Future work may involve adding post-processing modules for vehicle color and type recognition or integrating the detector into a full multi-object tracking (MOT) system.

Summary updated

Abstract

We perform fast vehicle detection from traffic surveillance cameras. A novel deep learning framework, namely Evolving Boxes, is developed that proposes and refines the object boxes under different feature representations. Specifically, our framework is embedded with a light-weight proposal network to generate initial anchor boxes as well as to early discard unlikely regions; a fine-turning network produces detailed features for these candidate boxes. We show intriguingly that by applying different feature fusion techniques, the initial boxes can be refined for both localization and recognition. We evaluate our network on the recent DETRAC benchmark and obtain a significant improvement over the state-of-the-art Faster RCNN by 9.5% mAP. Further, our network achieves 9-13 FPS detection speed on a moderate commercial GPU.
BibTeX
@inproceedings{wang2017evolving,
  author = {Li Wang and Yao Lu and Hong Wang and Yingbin Zheng and Hao Ye and Xiangyang Xue},
  title = {{Evolving Boxes for fast Vehicle Detection}},
  booktitle = {ICME},
  year = {2017},
  url = {https://openreview.net/forum?id=yVv9ql0HyTn},
}
Figure