Real-time Traffic Object Detection for Autonomous Driving

Abdul Hannan Khan, Syed Tahseen Raza Rizvi, Andreas DengelarXiv 2024 · DOI · arXiv

Introduction

In autonomous driving (AD), perception is the foundational step for safe navigation. While computer vision has significantly improved mean Average Precision (mAP), these gains often come with high computational costs. This paper addresses two critical deployment challenges: the "Energy Tax," where heavy models reduce vehicle range due to high power consumption, and the "Latency Gap," where slow processing speeds result in significant travel distance before an obstacle is detected. We propose an extension of the Localized Semantic Feature Mixers (LSFM) architecture to achieve true real-time performance (30+ FPS) without compromising accuracy.

Methodology

LSFM Architecture

Building on the LSFM framework originally designed for pedestrian detection, our model utilizes a ConvMLP-Pin backbone. This architecture extracts features and aligns them into patches, using MLPMixer blocks to efficiently mix spatial and channel information. This approach combines the representative power of Transformers with the computational efficiency of MLPs.

Multi-Class Extension and RTOP Metric

To generalize the model for traffic scenes (cars, trucks, motorcycles), we introduced a normalized center loss that calculates loss per class independently. Furthermore, we propose RTOP (Real-Time Objective Performance), a new evaluation metric. Unlike standard mAP, RTOP weights accuracy based on a target frame rate (e.g., 30 FPS), exponentially penalizing models that fail to meet real-time latency requirements.

Experimental Setup

We evaluated LSFM across several diverse benchmarks:

  • NuImages & BDD100K: Large-scale datasets with 10 object classes and varied environmental conditions.
  • TJU-DHD-Traffic: A high-resolution dataset focused on complex traffic scenes.
  • Euro City Persons: Used specifically for establishing baselines in nighttime and low-light conditions.
  • Shift: A synthetic dataset for testing resilience to continuous domain shifts like weather changes.

Key Results

Speed and Accuracy Trade-offs

Our evaluation demonstrates that LSFM variants provide a superior balance between latency and precision. The "LSFM P" (Performance) variant was the only model tested that consistently exceeded the 30 FPS threshold across all datasets. On the TJU-DHD-Traffic dataset, the "LSFM B" (Base) variant achieved a 60.4 mAP, outperforming Cascade R-CNN (57.9), FCOS (53.8), and YOLOv3 (56.8).

Figure 1: Comparison of mAP vs. inference latency (ms) across various datasets. LSFM P consistently achieves real-time performance (under ~33ms) while maintaining high accuracy.

Nighttime and Pedestrian Detection

LSFM shows remarkable resilience to motion blur and low-light conditions. On the Euro City Persons night dataset, LSFM achieved a Miss Rate (mMR) of 14.2, significantly outperforming Faster R-CNN (42.0) and matching the performance of much heavier architectures like SPNet while maintaining superior inference speeds.

Figure 4: Performance comparison on pedestrian detection benchmarks. LSFM achieves the fastest inference time (0.17s) and competitive miss rates (mMR) compared to state-of-the-art models.

Conclusion

LSFM bridges the gap between high-accuracy research models and the practical requirements of real-time autonomous systems. By achieving 30+ FPS and superior mAP on major benchmarks, it proves that efficiency and robustness can coexist. Future work will explore extending this lightweight architecture to 3D object detection and multi-sensor fusion pipelines.

Summary updated

BibTeX
@misc{khan2024real,
  author = {Abdul Hannan Khan and Syed Tahseen Raza Rizvi and Andreas Dengel},
  title = {{Real-time Traffic Object Detection for Autonomous Driving}},
  year = {2024},
  eprint = {2402.00128},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.00128},
}
Figure