Abstract
LiDAR-based 3D object detectors are critical components of autonomous vehicle perception systems. However, they are vulnerable to adversarial attacks that manipulate point cloud data through shifts, removals, or additions. This study provides a comprehensive benchmark of these vulnerabilities across eight state-of-the-art architectures. We introduce Balanced Adversarial Focal Training (BAFT), a novel defense mechanism that enhances model robustness without compromising accuracy on clean data.
Introduction
While deep learning has significantly advanced 3D object detection, safety-critical applications like autonomous driving require models to be robust against adversarial perturbations. In 2D vision, adversarial examples are well-documented, but their impact on 3D detectors—which must both classify and localize objects in sparse point clouds—is less understood. This research systematically evaluates the breaking points of leading detectors to establish a baseline for 3D adversarial robustness.
Adversarial Attack Paradigms
The study extends three primary attack methods to the 3D detection domain:
- Point Perturbation: Gradient-based methods like Projected Gradient Descent (PGD) are used to shift $(x, y, z)$ coordinates within a small threshold (e.g., 5–10cm).
- Point Detachment: Saliency maps identify "critical" points whose removal most significantly degrades detection performance.
- Point Attachment: The addition of "spoof" points to create false positives or obscure real objects.
Defense Strategies and BAFT
Beyond evaluating existing defenses like coordinate quantization and random sampling, the authors propose Balanced Adversarial Focal Training (BAFT). Standard adversarial training often suffers from a performance trade-off on clean data. BAFT addresses this by using focal weights to harmonize clean and adversarial losses, allowing the model to learn robust features without losing precision on standard benchmarks.
Experimental Results
The benchmark utilized the KITTI and Waymo Open Datasets to test eight detectors. Key findings include:
- Architecture Sensitivity: Voxel-based detectors are generally more robust than point-based ones due to the inherent "shielding" effect of voxelization, which dampens small coordinate shifts. PointPillar was found to be particularly fragile, with its mAP on KITTI dropping from 67.22% to 4.89% under a 10cm perturbation.
- BAFT Performance: BAFT outperformed standard adversarial training. For the SECOND detector, BAFT improved clean accuracy to 70.35% (up from 69.64%) while maintaining high adversarial robustness (77.27% adversarial mAP).
- Transferability: Attacks designed for one model often successfully fool others, particularly when transferring from point-based to voxel-based architectures.
Conclusion
This work highlights significant vulnerabilities in current LiDAR-based detectors and provides a path forward with BAFT. While digital attacks are potent, future research must address physically realizable adversarial objects and improve the computational efficiency of robust training methods.
Summary updated