Abstract
Advanced Driver-Assistant Systems (ADAS) often utilize fisheye cameras for their ultra-wide field of view, which is ideal for tasks like panorama stitching. However, the resulting geometric distortion significantly impairs the accuracy of standard object detection algorithms. This paper introduces MLSSD (Metric Learning-based Single Shot Multibox Detector), a specialized framework designed to detect vehicles and estimate distances directly from distorted fisheye footage. By employing multi-scale image partitioning and a dual-branch network architecture with metric learning, the proposed system achieves a 90.5% mean Average Precision (mAP) on distorted datasets, significantly outperforming standard models like YOLO and Faster R-CNN.
Introduction
The primary challenge in applying deep learning to fisheye imagery is the "warping" effect. Standard detectors like SSD or YOLO are typically trained on rectilinear images. When these models encounter the curved shapes of vehicles in a fisheye lens, their performance drops. Furthermore, there is a scarcity of large-scale, publicly available distorted datasets. This research bridges the gap by developing a model that learns distorted features directly, avoiding the computational cost and potential artifacts of image de-warping.
Proposed Methodology
The MLSSD framework utilizes a three-pronged approach to handle distortion effectively: multi-scale preprocessing, a metric learning-enhanced architecture, and a specialized pooling layer.
Multi-Scale Partition Preprocessing
To address the varying scales and levels of distortion across the fisheye field of view, the system crops the input image into six overlapping blocks at four different scales. This ensures that small, distant vehicles are "enlarged" in specific blocks, while larger, nearby vehicles are captured at an appropriate scale for the network.
Metric Learning-Based SSD (MLSSD)
The core of the system is a dual-branch architecture built upon a VGG-16 backbone. While one branch performs standard SSD detection, the second branch—the Metric Learning-based Classification (MLC) branch—is introduced to improve feature separation.
The MLC branch employs a "metric loss" function designed to minimize the distance between features of the same class (intra-class) and maximize the distance between different classes (inter-class). This makes vehicles much more distinguishable from the complex, warped backgrounds characteristic of fisheye lenses.
Scalable Overlapping Partition Pooling (SOPP)
Standard spatial pyramid pooling can lose critical spatial relationships between adjacent features. The authors introduce SOPP, which uses a sliding window approach with overlaps to transform feature maps of various sizes into a unified 16-dimensional vector. This preserves the spatial associations necessary for accurate detection in distorted environments.
Distance Estimation
Following detection, the system estimates distance using a "marker point" method. The bottom-center of a vehicle's bounding box is mapped to a grid of ground-truth marker points (spaced 30cm apart). Distance is then calculated via a "leaky Euclidean distance" algorithm, providing precise measurements for ADAS applications.
Experimental Results
The model was evaluated on a newly created Dataset A, containing 12,192 manually labeled distorted images, and the standard UA-DETRAC benchmark.
- Detection Accuracy: MLSSD achieved a 90.5% mAP on the distorted dataset, compared to 85.2% for standard SSD and 81.2% for YOLO. On the UA-DETRAC dataset, it maintained a robust 73.4% mAP.
- Distance Estimation: The system is highly accurate at close range, with an average error of only 0.08 meters for objects within 1 meter, and 0.33 meters for objects in the 2-3 meter range.
- Efficiency: Running on a GTX 1060, the model achieves 6.4 FPS. While slower than standard SSD (12 FPS) due to the multi-scale processing, the gain in accuracy for distorted imagery is substantial.
Conclusion
MLSSD demonstrates that focusing on feature space discriminability through metric learning is a powerful alternative to traditional image de-warping for fisheye-based ADAS. While current speeds are best suited for low-speed maneuvers like parking assistance, future work will focus on optimizing the network structure to achieve real-time performance and extending the reliable distance estimation range.
Summary updated