Robustness of Trajectory Prediction Models Under Map-Based Attacks

Zhihao Zheng, Xiaowen Ying, Zhen Yao, Mooi Choo ChuahWACV 2023

TL;DR

This paper introduces a novel attack strategy that targets the map information used by autonomous vehicle (AV) trajectory prediction models. By subtly altering context maps, the authors demonstrate significant degradation in prediction accuracy, highlighting a critical vulnerability in current AV systems. They also propose and evaluate defense mechanisms to mitigate these map-based attacks.

What Problem Does This Paper Address?

Autonomous vehicles (AVs) rely heavily on trajectory prediction (TP) to forecast the future movements of surrounding agents like cars and pedestrians. This prediction is crucial for safe navigation. While previous research has explored attacks on AV perception systems (e.g., object detection, depth estimation), this paper focuses on a less-explored vulnerability: the context map used by advanced TP models. Modern TP models often incorporate detailed map information (like lane layouts, drivable areas) to improve prediction accuracy. This paper investigates whether attackers can manipulate these context maps to mislead TP models, potentially leading to dangerous AV behavior. The challenge lies in crafting these map manipulations to be imperceptible to humans while still being effective against the AV's prediction algorithms.

Methods

The authors propose map-based adversarial attacks that target the context maps used by TP models. They categorize TP models based on how they encode map information:

  • Image-based encoding: The map is represented as a multi-channel image.
  • Node-based encoding: The map is represented as a set of nodes (e.g., lane centerlines).

For each category, they develop attack methods:

  • White-box attacks: These leverage gradient-based methods like Projected Gradient Descent (PGD) for continuous perturbations and a modified iterative gradient descent for binary perturbations. They aim to maximize prediction errors (ADE and FDE) by adding imperceptible changes to the map.
  • Black-box attacks: For models where gradients are not accessible, they employ Particle Swarm Optimization (PSO). PSO searches for optimal perturbations by treating candidate solutions as particles.

They also propose two defense mechanisms:

  • Image-based defense: A perturbation filter using morphological transformations (opening and closing) is applied before the map encoder.
  • Node-based defense: A convolution-based smoother is used to smooth node-based map representations, inspired by the effectiveness of 1D-CNN layers in some models.

Datasets

The primary dataset used for evaluation is nuScenes [1]. The authors use the validation set of nuScenes, which contains 146 scenarios, 3076 time frames, and 9040 agents. They also create a small subset of 10 scenarios for sensitivity analysis.

Key Insights and Ideas

  • Context maps are a critical vulnerability: Even sophisticated trajectory prediction models that leverage rich map information are susceptible to attacks that subtly alter these maps.
  • Imperceptible attacks are effective: The proposed attacks can significantly degrade prediction performance without being visually obvious to human observers.
  • Different map encodings require different attack strategies: Image-based maps are more susceptible to continuous perturbations, while node-based maps can be attacked by perturbing spatial coordinates. Binary perturbations are also explored for image-based maps to enhance robustness.
  • Black-box attacks are feasible: Even without full model knowledge, attackers can still launch effective map-based attacks using methods like PSO.
  • Defense mechanisms can mitigate attacks: Simple filtering and smoothing techniques can significantly reduce the impact of these map-based adversarial attacks.

Key Results

The map-based attacks significantly degrade the performance of state-of-the-art TP models:

  • Continuous white-box attacks increased average ADE/FDE by 83%/109% for image-based models and 55%/98% for node-based models.
  • Binary white-box attacks increased average ADE/FDE by 50%/67% for image-based models.
  • Black-box attacks showed weaker performance but were still effective, increasing average ADE/FDE by 16%/27% for image-based models and 24%/54% for node-based models.
  • The proposed defense mechanisms reduced the overall increase in average ADE/FDE to less than 5% for image-based models and by 25% for node-based models.
  • Qualitative results show that attacks can cause models to predict incorrect lane changes and wrong turns, highlighting potential safety risks.

Limitations and Future Work

  • Computational Cost: Generating adversarial perturbations, especially for black-box attacks using PSO, can be computationally intensive.
  • Specific Model Architectures: The effectiveness of attacks and defenses can vary depending on the specific architecture of the TP model and its map encoder. For instance, the 1D-CNN layer in LaPred improved its robustness.
  • Future Work: The authors suggest exploring more sophisticated defense mechanisms and investigating attacks on other components of the AV system in conjunction with map-based attacks. They also highlight the need for further research into the robustness of TP models against various types of adversarial attacks.

Summary updated

BibTeX
@inproceedings{zheng2023robustness,
  author = {Zhihao Zheng and Xiaowen Ying and Zhen Yao and Mooi Choo Chuah},
  title = {{Robustness of Trajectory Prediction Models Under Map-Based Attacks}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure