Evolving boxes for fast vehicle detection

Li Wang, Yao Lu, Hong Wang, Yingbin Zheng, Hao Ye, Xiangyang Xue

TL;DR

The "Evolving Boxes" (EB) framework introduces a multi-stage deep learning approach that progressively refines vehicle detection candidates through a sequence of specialized neural networks. By combining a light-weight proposal network with a sophisticated fine-tuning network and leveraging multi-layer feature fusion, the authors achieve a 9.5% mAP improvement over Faster R-CNN on the DETRAC benchmark while maintaining a practical speed of 9-13 FPS.

What Problem Does This Paper Address?

Vehicle detection in real-world traffic surveillance is notoriously difficult due to extreme variations in lighting, weather, vehicle pose, and frequent occlusions. While general-purpose detectors like Faster R-CNN have set high standards, they often struggle with the specific constraints of surveillance: the need to detect very small objects (vehicles far from the camera) and the requirement for high efficiency in processing dense video streams.

The authors identify a gap in how current models handle "candidate boxes." Instead of regressing a box once from a single set of features, they propose that boxes should "evolve"—being filtered and refined multiple times using different levels of visual information.

Methods

The EB framework consists of three primary components integrated into a single end-to-end pipeline:

  1. Deep Convolutional Network (DCN): Acts as the backbone (based on VGG-16) to extract rich feature representations from the input image.
  2. Proposal Network (PN): A light-weight stage that generates initial "anchor boxes." Its primary job is efficiency; it quickly discards roughly 98% of background regions that clearly don't contain vehicles, passing only high-probability candidates to the next stage.
  3. Fine-Tuning Network (FTN): This stage performs the heavy lifting. It takes the candidates from the PN and applies more sophisticated convolutional layers. Crucially, it uses Feature Fusion, concatenating features from the PN and the FTN to produce the final "evolved" bounding box and classification score.

The system is trained using a multi-stage loss function that balances the classification and localization errors of both the PN and FTN simultaneously.

Datasets

The researchers evaluated their framework using the DETRAC (Detection and Tracking) benchmark.

  • Scale: 140K captured frames with 1.2M labeled vehicles.
  • Diversity: Data was captured at various locations in Beijing under different conditions: sunny, cloudy, rainy, and night.
  • Challenge: The dataset is known for high vehicle density (averaging 8.6 vehicles per frame) and frequent occlusions.

Key Insights and Ideas

  • Evolutionary Refinement: The core "aha" moment is that localization shouldn't be a one-shot process. By allowing the PN to provide a "rough draft" and the FTN to "edit" that draft using higher-resolution features, the final boxes are significantly more accurate.
  • The Power of Low-Level Features: The authors found that concatenating features from earlier convolutional layers (1st, 3rd, and 5th) was critical. These lower-level layers retain spatial details that are often lost in deeper layers, which is essential for detecting small vehicles in the distance.
  • Early Discarding: By using a light-weight PN to filter out the majority of the background, the model avoids wasting expensive computational resources (the FTN) on empty road or sidewalk regions.

Key Results

  • Accuracy: On the DETRAC test set, the Evolving Boxes (EB) model achieved a 67.96% mAP, significantly outperforming Faster R-CNN (58.45%) and other models like CompACT (53.23%) and R-CNN (48.95%).
  • Sub-category Performance: EB performed best across all tested scenarios, including "Easy," "Medium," "Hard," and various weather conditions.
  • Ablation Gains: The inclusion of multi-layer feature fusion alone provided a massive 10.5% mAP boost, proving that spatial detail is king in surveillance tasks.
  • Efficiency: The model runs at 9-13 FPS (approx. 110ms per frame) on a moderate commercial GPU (Nvidia Titan X Maxwell), making it viable for near-real-time applications.

Limitations and Future Work

While the results are impressive, the authors noted specific failure cases:

  • Box Splitting: Occasionally, the detector splits a single large vehicle into multiple smaller bounding boxes.
  • Adjacent Vehicles: The model sometimes fails to distinguish between two vehicles that are very close to each other or overlapping.
  • Future Directions: The authors suggest that the high-quality detection results from EB could be further improved by adding post-processing modules specifically for vehicle color and type recognition, or by integrating the detector into a full multi-object tracking (MOT) system.

Summary updated

Abstract

We perform fast vehicle detection from traffic surveillance cameras. A novel deep learning framework, namely Evolving Boxes, is developed that proposes and refines the object boxes under different feature representations. Specifically, our framework is embedded with a light-weight proposal network to generate initial anchor boxes as well as to early discard unlikely regions; a fine-turning network produces detailed features for these candidate boxes. We show intriguingly that by applying different feature fusion techniques, the initial boxes can be refined for both localization and recognition. We evaluate our network on the recent DETRAC benchmark and obtain a significant improvement over the state-of-the-art Faster RCNN by 9.5% mAP. Further, our network achieves 9–13 FPS detection speed on a moderate commercial GPU.
BibTeX
@inproceedings{wang2017evolving,
  author = {Li Wang and Yao Lu and Hong Wang and Yingbin Zheng and Hao Ye and Xiangyang Xue},
  title = {{Evolving boxes for fast vehicle detection}},
  booktitle = {ICME},
  year = {2017},
  url = {https://openreview.net/forum?id=yVv9ql0HyTn},
}

Loading related content...

Figure