TL;DR
This paper introduces a novel "safety-adapted focal loss" for pedestrian detection in autonomous driving. By incorporating a "criticality score" for each pedestrian, derived from their predicted collision risk and proximity to the vehicle, the model learns to prioritize detecting pedestrians who pose the greatest danger, thereby reducing critical false negatives without compromising overall detection performance.
What Problem Does This Paper Address?
In automated driving (AD), accurately detecting vulnerable road users (VRUs) like pedestrians is paramount for safety. Traditional object detection metrics, such as mean average precision (mAP), don't adequately capture the safety implications of misdetections. A false negative for a pedestrian in the direct path of an autonomous vehicle (AV) is far more dangerous than one for a pedestrian far away. Current training methods penalize all misdetections equally, failing to prioritize the detection of "safety-critical" pedestrians. This paper aims to bridge this gap by developing a training strategy that explicitly accounts for the varying levels of risk posed by individual pedestrians.
Methods
The core contribution is a novel safety-adapted focal loss function. This loss function modifies the standard focal loss by incorporating a per-pedestrian criticality score (Kᵢ). This score is calculated by combining two factors:
- Collision Criticality (K<0xE1><0xB5><0x9C>,ᵢ): Derived from the Reachability Set-Based Time-to-Collision (TTCRSB). This metric uses reachability analysis from the motion domain to estimate the earliest possible time a collision could occur between the AV and a pedestrian, considering their uncertain future movements.
- Distance Criticality (K<0xE1><0xB5><0x9A>,ᵢ): Incorporates the physical distance between the pedestrian and the AV. This is crucial because even if the TTCRSB is not immediately critical, a pedestrian close to the AV still poses a significant risk.
The safety-adapted focal loss then uses this criticality score (Kᵢ) to dynamically adjust the loss contribution for each pedestrian. Critically, pedestrians with higher criticality scores have their misdetections penalized more heavily during training, effectively guiding the neural network to focus on them. The authors also propose a specific formulation for Kᵢ, weighting collision criticality more heavily and using a downward parabola for distance criticality to ensure that proximity is strongly considered.
Datasets
- nuImages: Used for initial pre-training of the pedestrian detectors. This dataset provides 2D bounding box annotations.
- nuScenes: The primary dataset for training and evaluation. It's a multimodal dataset providing 3D sensor data (including lidar and camera) and detailed annotations for 1000 scenes. The authors project 3D bounding boxes from nuScenes onto the camera view to obtain 2D pedestrian annotations.
Key Insights and Ideas
- Beyond Generic Metrics: Standard evaluation metrics are insufficient for safety-critical applications like AD. We need metrics that reflect the actual risk posed by misdetections.
- Motion Domain for Safety: Integrating information from the motion domain (via reachability analysis and TTCRSB) is crucial for accurately assessing collision risk, especially when considering uncertainties in pedestrian and AV movement.
- Proximity Matters: Even with a seemingly safe TTCRSB, a pedestrian's close proximity to the AV significantly increases their risk, necessitating a combined criticality score.
- Instance-Level Safety: Safety is not a category-wide property but an instance-specific one. Different pedestrians in the same scene can have vastly different risk profiles.
- Targeted Training: By adapting the loss function based on individual pedestrian criticality, we can effectively "teach" the model to prioritize the detection of the most dangerous scenarios.
Key Results
The paper demonstrates that training with the safety-adapted focal loss (FL<0xE2><0x82><0x96>) leads to significant improvements in recall for safety-critical pedestrians (Zone C).
- For RetinaNet, recall in Zone C improved by 2.5% compared to the baseline focal loss with γ=2.
- For FCOS, recall in Zone C improved by 3.2% compared to the baseline focal loss with γ=2.
- The safety-adapted loss also showed improvements for potentially critical pedestrians (Zone PC) and maintained stable overall performance, with a negligible degradation of less than 1.0% in overall performance metrics.
- An ablation study showed that while distance alone can be a useful proxy for criticality, combining it with collision risk (TTCRSB) provides a more robust measure.
Limitations and Future Work
- False Positives: The current work focuses on mitigating false negatives (missed detections). The authors acknowledge that false positives are also a significant safety concern and suggest future work to determine the criticality of falsely detected pedestrians.
- Regression Task: The current safety adaptation is applied to the detection loss. Future work aims to extend this to the regression task to improve the localization accuracy of critical pedestrians.
- Visibility Proxy: While visibility was used as a proxy for learning difficulty in an ablation study, it's a simplified measure. More sophisticated difficulty metrics could be explored.
- Model Dependence: The ablation study indicated that the effectiveness of simple distance-based criticality might vary between models, suggesting that the proposed combined criticality is more generally applicable.
Summary updated