Affine Disentangled GAN for Interpretable and Robust AV Perception

Letao Liu, Martin Saerbeck, Justin DauwelsarXiv 2019 · arXiv

Autonomous vehicle perception systems must be resilient to environmental changes and malicious attacks to ensure passenger safety.

Introduction

Deep learning models, particularly Convolutional Neural Networks (CNNs), serve as the primary sensory organs for autonomous vehicles (AVs). However, these models exhibit significant vulnerabilities that can compromise safety in real-world environments. Researchers have introduced the Affine Disentangled GAN (ADIS-GAN) to address these weaknesses. This novel architecture enhances robustness against both malicious adversarial attacks and common geometric distortions, providing a more interpretable framework for AV perception.

The Vulnerability Gap in AV Perception

Modern AV perception systems face two primary threats: adversarial attacks and affine transformations. Adversarial attacks involve subtle, human-invisible pixel perturbations that can trick a classifier into making dangerous errors, such as misidentifying a stop sign. Simultaneously, simple affine transformations—such as the rotation or scaling that occurs when a vehicle tilts on a bumpy road—can cause state-of-the-art models to fail.

An illustration of the 'Achilles' heel' of standard classifiers: simple rotations can lead a model to misidentify a vehicle as a 'Moving Van', 'Snowplow', or 'School Bus'.

A critical observation made by the authors is the "orthogonality" of these threats. Standard data augmentation (training on rotated images) typically fails to defend against adversarial noise, and adversarial training does not inherently improve robustness to rotations. ADIS-GAN is designed to bridge this gap by treating these issues as interconnected challenges rather than isolated problems.

Methodology: Disentanglement and Regularization

The ADIS-GAN architecture evolves from several foundational generative models to create a robust "generative defense" mechanism:

  • Bi-directional GAN: This allows the model to map real images into a latent space and reconstruct them. By passing an input through this encoding-decoding loop, the system naturally strips away malicious adversarial noise before classification occurs.
  • InfoGAN Principles: These are used to "disentangle" the latent space, ensuring that specific internal variables correspond to distinct, human-interpretable physical features.
  • The Affine Regularizer: This is the paper's core innovation. By mathematically decomposing affine transformation matrices into specific rotation and zoom components, the authors force the GAN to maximize mutual information between latent codes and these physical transformations.

This approach creates "interpretable knobs"—specific latent vectors that directly control or represent physical properties like rotation angle, moving the model away from being an opaque "black box."

Experimental Insights and Performance

The researchers validated ADIS-GAN using the MNIST benchmark to demonstrate its ability to handle synthetic rotations and adversarial perturbations generated by tools like FoolBox.

Key Results:

  • Rotation Robustness: The model maintained over 98% classification accuracy across rotation angles ranging from -30 to +30 degrees, matching the performance of models trained exclusively for rotation.
  • Adversarial Defense: ADIS-GAN achieved 93.10% accuracy against FGSM (Fast Gradient Sign Method) attacks and 96.53% against the more powerful PGD (Projected Gradient Descent) attacks. In stark contrast, a standard model's accuracy plummeted to a dismal 0.10% under PGD conditions.
  • Interpretability: The disentangled latent space showed a clear linear relationship with physical rotation, allowing engineers to inspect the model's internal state to understand exactly why a classification decision was made.

Conclusion and Future Directions

ADIS-GAN represents a significant step toward more resilient and transparent AV perception. By combining generative defense with affine regularization, it provides dual-layered protection against the most common failure modes of deep learning in the field.

While the current proof-of-concept focused on grayscale datasets, future work will focus on scaling this architecture to high-resolution RGB datasets like Cityscapes and expanding the regularizer to handle more complex 3D perspective shifts and shearing. Improving the training stability of these bi-directional components remains a priority for deploying such systems in real-time automotive hardware.

Summary updated

Abstract

Autonomous vehicles (AV) have progressed rapidly with the advancements in computer vision algorithms. The deep convolutional neural network as the main contributor to this advancement has boosted the classification accuracy dramatically. However, the discovery of adversarial examples reveals the generalization gap between dataset and the real world. Furthermore, affine transformations may also confuse computer vision based object detectors. The degradation of the perception system is undesirable for safety critical systems such as autonomous vehicles. In this paper, a deep learning system is proposed: Affine Disentangled GAN (ADIS-GAN), which is robust against affine transformations and adversarial attacks. It is demonstrated that conventional data augmentation for affine transformation and adversarial attacks are orthogonal, while ADIS-GAN can handle both attacks at the same time. Useful information such as image rotation angle and scaling factor are also generated in ADIS-GAN. On MNIST dataset, ADIS-GAN can achieve over 98 percent classification accuracy within 30 degrees rotation, and over 90 percent classification accuracy against FGSM and PGD adversarial attack.
BibTeX
@misc{liu2019affine,
  author = {Letao Liu and Martin Saerbeck and Justin Dauwels},
  title = {{Affine Disentangled GAN for Interpretable and Robust AV Perception}},
  year = {2019},
  eprint = {1907.05274},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1907.05274},
}
Figure