Abstract
Monitoring urban intersections effectively often requires wide-angle coverage, typically achieved through 360-degree fisheye cameras. However, these cameras introduce significant non-linear barrel distortion and face extreme lighting variations. This paper presents an optimized YOLOv5-based pipeline designed for real-time vehicle detection in these challenging environments. By integrating a day-night image routing classifier, a multistage transfer learning strategy, and selective model ensembling, the proposed approach achieves a 13.7% improvement in [email protected] over the baseline YOLOv5 while maintaining real-time performance.
Introduction
Urban traffic surveillance is critical for smart city infrastructure. While 360-degree fisheye cameras offer a cost-effective way to monitor entire intersections with a single device, they present unique computer vision challenges. Vehicles at the periphery of the fisheye lens undergo severe distortion, appearing significantly different from those in the center. Additionally, environmental factors such as nighttime glare, low-light noise, and harsh daytime shadows further complicate detection. Standard object detection models often struggle to generalize across these distortions and lighting extremes. This work aims to bridge this gap by optimizing the YOLOv5 architecture for robust, real-time intersection monitoring.
Methodology
The proposed system leverages the YOLOv5 architecture as its core detection engine. A key feature of YOLOv5 is the "Focus block," which spatially squeezes pixels to preserve inter-pixel relationships, aiding the network in interpreting non-linear distortions at the edges of fisheye images.
To address the specific challenges of fisheye surveillance, the authors implemented a comprehensive training and inference pipeline:
- Day-Night Separator: A lightweight CNN classifies incoming frames as "day" or "night," routing them to specialized models optimized for specific lighting conditions.
- Multistage Transfer Learning: To avoid the harsh gradient shift from general objects to distorted fisheye data, weights are first pre-trained on COCO, then fine-tuned on the BDD100k dashcam dataset to learn vehicle-specific features, and finally trained on the target fisheye data.
- Data Augmentation and Pseudo-Labeling: An external dataset of unlabeled fisheye videos was used to generate "pseudo-labels," enriching the training pool. Furthermore, "challenging" images with low mAP scores were oversampled by 10x to improve model robustness on edge cases.
- Selective Ensembling: Final predictions are generated by merging bounding boxes from multiple models trained on different sequential partitions of the data.
Experimental Setup
The primary evaluation was conducted on the IEEE VIP Cup 2020 Dataset, which contains approximately 25,000 daytime and 10,000 nighttime fisheye images. Supplemental datasets included COCO for initial pre-training and BDD100k for intermediate transfer learning. A custom Pseudo Dataset was also created from surveillance videos in Thailand to further diversify the training data.
Results and Discussion
The optimized pipeline significantly outperformed the baseline YOLOv5, achieving a 13.7% increase in [email protected] on the VIP Cup dataset. The Day-Night separator achieved 100% classification accuracy, ensuring reliable routing. Nighttime performance reached a peak of 0.767 [email protected] using an ensemble of models at 768x768 resolution.
Qualitative analysis showed that the ensembled model effectively resolved complex structures, such as correctly identifying articulated trams as single vehicles and detecting cars in deep shadows or at extreme lens edges. Despite these additions, the system maintains real-time inference speeds, with the core YOLOv5s model processing frames in approximately 30ms.
An interesting observation was made regarding resolution: increasing input resolution improved Average Recall (AR) but decreased Average Precision (AP). Investigation revealed this was due to the model detecting tiny, distant vehicles missed by human annotators, resulting in "false positives" that were actually correct detections.
Conclusion
This research demonstrates that a combination of specialized routing, multistage transfer learning, and selective ensembling can overcome the inherent difficulties of fisheye-based vehicle detection. By tailoring the YOLOv5 pipeline to handle specific environmental and geometric challenges, the proposed system provides a robust solution for real-time urban traffic monitoring. Future work will focus on closing the performance gap between day and night detection by incorporating more diverse nighttime pseudo-data.
Summary updated