TL;DR
This paper demonstrates that Tracking-By-Detection (TBD) systems in autonomous vehicles are highly vulnerable to adversarial perturbations, which can cause tracking accuracy to plummet or even reach negative values. By evaluating both image-specific and universal adversarial attacks, the researchers show that while newer detectors like YOLOv4 offer slightly better robustness, the entire perception pipeline remains susceptible to "ghost" objects and fragmented trajectories.
What Problem Does This Paper Address?
Autonomous vehicles (AVs) rely on Multi-Object Tracking (MOT) to understand the movement of surrounding traffic. Most modern systems use a Tracking-By-Detection (TBD) approach: a neural network detects objects in each frame, and a tracking algorithm links these detections over time to form trajectories.
While we know that object detectors are easily fooled by adversarial noise, it was previously unclear how these errors propagate through the tracking stage. Does the temporal logic of a tracker (like a Kalman filter) act as a natural defense by filtering out transient detection errors? This paper proves that persistent adversarial attacks can overwhelm these safeguards, leading to dangerous failures like fragmented tracks and a flood of spurious "false positive" detections.
Methods
The authors evaluate the robustness of the TBD framework by attacking the detector component. They focus on two types of attacks:
- Image-Specific Attacks: A unique perturbation is calculated for every single video frame to maximize a custom adversarial loss function ($L_{adv}$).
- Image-Agnostic Attacks: A single Universal Adversarial Perturbation (UAP) is learned from a training set and applied statically to every frame in a sequence. This is computationally cheaper and more representative of a "set-and-forget" physical attack.
The adversarial loss function is specifically designed to encourage the detector to produce many high-confidence but small bounding boxes, effectively "drowning" the tracker in false positives.
The study evaluates these attacks across a combination of:
- Detectors: YOLOv3, YOLOv4, and YOLOX.
- Trackers: SORT, Deep SORT, Strong SORT, and OC-SORT.
Datasets
The researchers developed and open-sourced a framework to generate synthetic autonomous driving datasets using the CARLA Simulator. This allowed them to test across seven different maps with diverse environmental conditions and perfect ground-truth data for evaluation.
Key Insights and Ideas
- The False Positive Flood: The most effective way to break a tracker isn't necessarily to make objects "disappear," but to fill the scene with high-confidence "ghost" detections. This forces the association algorithm (like the Hungarian algorithm) to make incorrect matches, leading to a total collapse of trajectory estimation.
- YOLOv4's Relative Robustness: YOLOv4 proved more resilient than YOLOv3. While YOLOv3 produced thousands of tiny false positives (leading to massive negative MOTA scores), YOLOv4 produced fewer, larger spurious boxes. This suggests that architectural improvements in detectors do provide some inherent protection, though not enough to be considered "secure."
- Association is Not a Defense: A common hope was that the Kalman filter's motion prediction would reject adversarial detections as "noise." However, the study shows that if the detector is consistently fooled, the tracker eventually accepts the false data as a valid trajectory.
- UAP Effectiveness: Universal perturbations (UAPs) are surprisingly effective. Even though they aren't tailored to specific frames, they can still significantly degrade tracking metrics, making them a viable threat for real-world deployment.
Key Results
The quantitative impact of these attacks is staggering:
- YOLOv3 Collapse: Under image-specific attacks, the Multiple Object Tracking Accuracy (MOTA) for YOLOv3-based systems dropped from ~60% to as low as -772.60%. (MOTA can be negative because it penalizes false positives without a lower bound).
- YOLOv4 Degradation: YOLOv4 was more robust but still saw MOTA drop from ~65% to between 0.94% and 19.19%.
- HOTA Metrics: The Higher Order Tracking Accuracy (HOTA), which balances detection and association, declined from ~55% to ~15% for YOLOv3 and ~25% for YOLOv4.
Limitations and Future Work
- Synthetic vs. Real World: The study uses synthetic data from CARLA. While CARLA is high-fidelity, real-world sensor noise and atmospheric conditions might interact with adversarial perturbations differently.
- Class Filtering: The authors noted that many adversarial detections were labeled as "birds" or "chairs." In a real AV system, a simple class filter (only tracking "cars" or "pedestrians") might mitigate some of the impact, though vehicle-targeted attacks would likely bypass this.
- Defense Mechanisms: The paper focuses on the attack. Future work is needed to develop robust association modules or adversarial training techniques specifically for the temporal nature of tracking.
Summary updated