A Little Fog for a Large Turn

Harshitha Machiraju, Vineeth N. Balasubramanian

TL;DR

This paper introduces a novel way to test the robustness of autonomous driving systems by generating "adversarial fog" using generative models. By creating realistic foggy images that subtly trick steering angle prediction models, they provide a more natural and effective method for evaluating system safety compared to traditional, often artificial, adversarial attacks.

What Problem Does This Paper Address?

Autonomous vehicles rely heavily on computer vision for tasks like steering angle prediction. While deep neural networks have achieved impressive performance, they are notoriously vulnerable to adversarial attacks – small, often imperceptible, perturbations to input data that can cause misclassifications or incorrect predictions. Traditional adversarial attacks are typically additive noise or carefully crafted patches, which aren't representative of real-world challenges. Adverse weather conditions, like fog, are known to significantly degrade the performance of these systems and can be thought of as "natural adversaries." However, existing methods for testing robustness against such conditions are either manual and labor-intensive (e.g., Photoshop) or lack a specific adversarial goal (e.g., simple image-to-image translation). This paper addresses the need for a systematic, automated method to generate realistic, adversarial weather conditions to rigorously test and improve the robustness of autonomous navigation systems.

Methods

The core of the proposed method is a generative approach inspired by Cycle-Consistent Generative Adversarial Networks (CycleGAN). The authors adapt CycleGAN to learn the translation between normal (sunny) weather images (Domain A) and foggy weather images (Domain B). Crucially, they augment the standard CycleGAN loss with a "regression loss." This regression loss is designed to specifically encourage the generated foggy images to cause a minimum deviation in the steering angle predicted by a target autonomous navigation model.

The authors define adversarial perturbations more broadly as "task-perceptual similarity," meaning the generated image should be perceived similarly by a human for the given task (steering angle prediction), even if it looks different to a neural network. They formalize this by minimizing the difference between the predicted steering angle of the original image and the generated foggy image. The overall training objective combines the CycleGAN's image-to-image translation capabilities with this adversarial regression loss.

Datasets

  • SullyChen dataset: Used for training the normal (sunny) weather domain images.
  • Foggy Zurich dataset: Used for training the foggy weather domain images and as a control for subjective image quality assessment.
  • Udacity dataset: Mentioned as an alternative but deemed less suitable due to image quality issues.

The authors note that their framework is generic and can be extended to other weather conditions if suitable datasets are available.

Key Insights and Ideas

  • Natural Adversaries: Fog is a powerful "natural adversary" that can significantly impact steering angle predictions, highlighting a critical vulnerability in current autonomous systems.
  • Task-Perceptual Similarity: The paper proposes a more intuitive definition of adversarial perturbations, focusing on how the image is perceived for the specific task, rather than just visual similarity. This allows for more naturalistic attacks.
  • Adversarial Regression: Extending adversarial attack concepts from classification to regression tasks (like steering angle prediction) is crucial for testing systems like autonomous vehicles.
  • Generative Adversarial Fog: Using CycleGAN to generate realistic foggy images that are specifically designed to fool steering models offers a powerful and automated way to create challenging test cases.

Key Results

  • The generated foggy images can cause significant deviations in steering angle predictions, with one example showing a nearly 180-degree change.
  • Subjective assessments showed that nearly 48% of participants found the generated fog to be realistic, comparable to the original Foggy Zurich dataset.
  • Adversarial training using the generated foggy images significantly improved the robustness of steering angle prediction models, reducing the deviation caused by fog from around 1.8 radians to approximately 0.2 radians.
  • The paper demonstrates that a balance between the image translation quality (CycleGAN loss) and the adversarial goal (regression loss) is important for generating effective and realistic adversarial examples.

Limitations and Future Work

  • Dataset Limitations: The current work is limited to foggy conditions due to the unavailability of large-scale datasets for other adverse weather conditions like heavy snow or rain.
  • Backward Cycle Distortion: Applying the regression loss to the backward cycle (foggy to sunny) resulted in distorted images, suggesting that the adversarial goal should primarily focus on generating adverse conditions, not necessarily making them perfectly reversible.
  • Future Work: The authors aim to construct more stringent testbeds by exploring other weather conditions and potentially developing more sophisticated generative models. They also emphasize the need for better datasets to further evaluate the credibility of autonomous navigation systems.

Summary updated

Abstract

Small, carefully crafted perturbations called adversarial perturbations can easily fool neural networks. However, these perturbations are largely additive and not naturally found. We turn our attention to the field of Autonomous navigation wherein adverse weather conditions such as fog have a drastic effect on the predictions of these systems. These weather conditions are capable of acting like natural adversaries that can help in testing models. To this end, we introduce a general notion of adversarial perturbations, which can be created using generative models and provide a methodology inspired by Cycle-Consistent Generative Adversarial Networks to generate adversarial weather conditions for a given image. Our formulation and results show that these images provide a suitable testbed for steering models used in Autonomous navigation models. Our work also presents a more natural and general definition of Adversarial perturbations based on Perceptual Similarity <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
BibTeX
@inproceedings{machiraju2020little,
  author = {Harshitha Machiraju and Vineeth N. Balasubramanian},
  title = {{A Little Fog for a Large Turn}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=WOLs-8CQh7},
}
Figure