RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation

Jiawei Zhou, Linye Lyu, Daojing He, Yu Li

Abstract

Deep Neural Networks (DNNs) serve as the cornerstone of modern autonomous vehicle perception systems. However, these systems are vulnerable to physical adversarial attacks, where specifically designed patterns applied to objects can cause detectors to fail. Existing physical camouflage methods often struggle with environmental variability and the "sim-to-real" gap. This paper introduces RAUCA (Robust and Accurate UV-map-based Camouflage Attack), a framework that utilizes a high-fidelity neural renderer and UV-map optimization to generate camouflage that remains effective across diverse weather and lighting conditions. Experimental results demonstrate that RAUCA significantly outperforms existing state-of-the-art methods in both simulated environments and real-world physical tests.

1. Introduction

The reliability of object detection models is paramount for the safety of autonomous driving. While digital adversarial attacks—modifying image pixels directly—have been extensively researched, they are impractical for real-world scenarios. Physical attacks, such as applying adversarial textures to a vehicle's surface, represent a more tangible threat. Despite their potential, current physical attacks face two primary challenges: environmental sensitivity, where changes in lighting or weather (fog, rain) neutralize the attack, and mapping inaccuracy, where the transition from a 2D digital design to a complex 3D physical surface degrades the adversarial effect.

2. The RAUCA Framework

RAUCA addresses these challenges through a multi-stage optimization process that bridges the gap between digital simulation and physical reality.

2.1 Neural Renderer Plus (NRP)

Standard differentiable renderers often fail to capture the complex interplay between an object's surface and its environment, leading to "flat" representations. RAUCA introduces the Neural Renderer Plus (NRP), which incorporates an Environment Feature Extractor (EFE). This encoder-decoder network learns to blend the 3D vehicle model with realistic environmental features, such as shadows and atmospheric scattering. By training the optimizer on these high-fidelity representations, the resulting camouflage is better prepared for the nuances of the physical world.

2.2 UV-Map Optimization

Rather than using "world-aligned" methods that simply tile a pattern over a vehicle's bounding box, RAUCA optimizes a UV map. This approach allows the adversarial texture to be precisely wrapped around the vehicle's specific geometry. This ensures that the adversarial features remain coherent and effective regardless of the camera's viewing angle or the vehicle's orientation.

2.3 Robust Loss Function

The framework employs a comprehensive loss function designed to minimize the detector's confidence scores, objectness scores, and Intersection over Union (IoU) metrics. By targeting these three pillars of detection simultaneously, RAUCA forces the DNN to either ignore the vehicle entirely or produce bounding boxes so inaccurate they are discarded by the system.

3. Experimental Methodology

3.1 Simulation Environment

The researchers utilized the CARLA autonomous driving simulator to generate a robust training and testing pipeline.

  • Multi-Weather Training: A dataset of 69,120 images was created, featuring 16 distinct weather conditions derived from varying sun altitudes and fog densities.
  • Generalization Testing: An "Unseen-Weather" dataset of 7,593 images (e.g., MidRainyNight) was used to evaluate how well the camouflage performs in conditions it did not encounter during optimization.

3.2 Physical Evaluation

To validate the "sim-to-real" transition, the authors printed the optimized textures and applied them to 1:12 scale Audi E-Tron models. Testing involved 552 physical photographs captured under various lighting conditions and simulated fog (generated via a smoke machine) to mimic the complexities of real-world environments.

4. Results and Discussion

4.1 Performance against SOTA

In white-box evaluations against YOLOv3, RAUCA demonstrated superior performance. While a standard vehicle achieved an Average Precision ([email protected]) of 0.70, RAUCA reduced this to 0.304. This significantly outperformed previous state-of-the-art methods like ACTIVE (~0.44), DAS, and FCA.

4.2 Black-Box Transferability

RAUCA exhibited strong transferability to other detection architectures, including YOLOX, DETR, and Faster R-CNN. In unseen weather conditions, RAUCA maintained an advantage of approximately 8-11% over competing methods, suggesting that the features learned through the NRP are more universally disruptive to neural feature extraction.

4.3 Real-World Robustness

Physical tests confirmed the simulation findings. RAUCA was uniquely robust across different times of day and atmospheric conditions. Notably, in night-time tests, RAUCA achieved an AP of 0.137, while competitors failed to drop the AP below 0.24, highlighting the effectiveness of the multi-weather training strategy.

5. Conclusion

RAUCA represents a significant advancement in physical adversarial research by demonstrating that high-fidelity rendering and precise geometric mapping are essential for robust attacks. While the current study utilized 1:12 scale models, the results suggest that such attacks could pose a credible threat to full-scale autonomous systems. Future work will focus on improving the "naturalness" of the camouflage to make it less conspicuous to human observers and exploring the computational efficiency of the UV-map optimization process.

Summary updated

BibTeX
@inproceedings{zhou2024rauca,
  author = {Jiawei Zhou and Linye Lyu and Daojing He and Yu Li},
  title = {{RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=pBTLGM9uWx},
}
Figure