Embodied Adversarial Attack: A Dynamic Robust Physical Attack in Autonomous Driving

Yitong Sun, Yao Huang, Xingxing Wei

Abstract

As physical adversarial attacks become extensively applied in unearthing the potential risk of security-critical scenarios, their vulnerability to environmental variations has also been brought to light. The non-robust nature of physical adversarial attack methods brings less-than-stable performance. Although methods such as Expectation over Transformation (EOT) have enhanced the robustness of traditional contact attacks, they fall short in dynamic traffic scenarios. Non-contact laser attacks face constraints due to a limited optimization space, rendering EOT less effective. This paper introduces the Embodied Laser Attack (ELA), a novel framework leveraging the embodied intelligence paradigm of Perception-Decision-Control to dynamically tailor non-contact laser attacks. ELA utilizes a local perspective transformation network for efficient victim-view estimation and a reinforcement learning agent for instantaneous strategy determination. Experimental results in both digital and physical worlds verify ELA's effectiveness under dynamic successive scenes.

Introduction

Deep Neural Networks (DNNs) in autonomous driving are vulnerable to physical adversarial attacks. While "contact" attacks like adversarial patches are effective, they are easily detected and hard to deploy secretly. "Non-contact" attacks, such as those using lasers or projectors, offer better concealment but are highly sensitive to changes in angle, distance, and lighting as a vehicle moves. Traditional robustness techniques like EOT struggle with the limited parameter space of lasers (color, width, angle). ELA shifts the focus from static robustness to active adaptation, using an embodied agent to adjust the attack in real-time as the vehicle moves through the scene.

Methodology: The ELA Framework

The ELA framework follows the "Perception-Decision-Control" paradigm to maintain attack effectiveness against a moving target.

The ELA framework pipeline, illustrating the Perception-Decision-Control flow and the resulting attack on the vehicle perception system.

1. Perception: Perspective Transformation Network (PTN)

To successfully attack, the system must estimate the victim vehicle's perspective. ELA employs a Perspective Transformation Network (PTN) that utilizes geometric priors. By treating the car as a rectangle and traffic signs as known shapes (circles or octagons), PTN maps the car's position from the attacker's roadside view to the sign's appearance in the victim's view. This approach achieves over 90% Intersection-over-Union (IoU) accuracy and is significantly more efficient than full-image transformation.

2. Decision and Control: RL Agent

The core of ELA is a Reinforcement Learning (RL) agent trained using Proximal Policy Optimization (PPO).

  • State: The current perceived state of the traffic sign and laser parameters.
  • Action: Real-time adjustments to the laser's slope, width, and wavelength.
  • Reward: The agent is optimized to maximize the probability of misclassification while maintaining a "natural" laser appearance (thin beams within the visible spectrum).

Experimental Evaluation

The system was tested using the CARLA simulator across various maps and the GTSRB dataset for victim models.

Comparison between static EOT and the dynamic ELA. ELA successfully adapts to the moving vehicle's perspective, maintaining a high attack success rate where static methods fail.

Key Results

  • Attack Success Rate (ASR): In dynamic scenarios, ELA achieved an ASR of up to 76%, vastly outperforming static methods like AdvLB, which dropped to 7-15% due to a lack of adaptation.
  • Efficiency: ELA generates new strategies in 0.13s to 0.22s, enabling real-time operation against moving vehicles.
  • Physical World Robustness: Tests in outdoor environments confirmed that ELA remains effective under varying weather and lighting conditions, with ASRs between 44% and 68%.

Conclusion

ELA demonstrates that embodied intelligence can significantly enhance the potency of physical adversarial attacks. By moving from static perturbations to dynamic, adaptive strategies, ELA exposes critical vulnerabilities in current autonomous driving perception systems. This research highlights the need for more robust, adaptive defense mechanisms that can account for dynamic, light-based adversarial interventions.

Summary updated

Abstract

As physical adversarial attacks become extensively applied in unearthing the potential risk of security-critical scenarios, especially in dynamic scenarios, their vulnerability to environmental variations has also been brought to light. The non-robust nature of physical adversarial attack methods brings less-than-stable performance consequently. Although methods such as EOT have enhanced the robustness of traditional contact attacks like adversarial patches, they fall short in practicality and concealment within dynamic environments such as traffic scenarios. Meanwhile, non-contact laser attacks, while offering enhanced adaptability, face constraints due to a limited optimization space for their attributes, rendering EOT less effective. This limitation underscores the necessity for developing a new strategy to augment the robustness of such practices. To address these issues, this paper introduces the Embodied Laser Attack (ELA), a novel framework that leverages the embodied intelligence paradigm of Perception-Decision-Control to dynamically tailor non-contact laser attacks. For the perception module, given the challenge of simulating the victim's view by full-image transformation, ELA has innovatively developed a local perspective transformation network, based on the intrinsic prior knowledge of traffic scenes and enables effective and efficient estimation. For the decision and control module, ELA trains an attack agent with data-driven reinforcement learning instead of adopting time-consuming heuristic algorithms, making it capable of instantaneously determining a valid attack strategy with the perceived information by well-designed rewards, which is then conducted by a controllable laser emitter. Experimentally, we apply our framework to diverse traffic scenarios both in the digital and physical world, verifying the effectiveness of our method under dynamic successive scenes.
BibTeX
@inproceedings{sun2023embodied,
  author = {Yitong Sun and Yao Huang and Xingxing Wei},
  title = {{Embodied Adversarial Attack: A Dynamic Robust Physical Attack in Autonomous Driving}},
  booktitle = {ACM Multimedia},
  year = {2023},
}
Figure