Understanding the Limitations of Conditional Generative Models

Ethan Fetaya, Jörn-Henrik Jacobsen, Will Grathwohl, Richard S. Zemel

While researchers hoped that class-conditional generative models would be naturally robust to adversarial attacks by modeling the entire input distribution, recent findings prove otherwise. This paper demonstrates a theoretical impossibility result for guaranteed robustness and shows that while these models work for simple data like MNIST, they fail catastrophically on complex datasets like CIFAR10 due to the fundamental nature of the maximum likelihood objective.

The Promise of Conditional Generative Models

Deep learning has a well-documented "fragility" problem: discriminative classifiers, which only model the conditional probability $p(y|x)$, are easily fooled by tiny, adversarial perturbations. A popular hypothesis was that Conditional Generative Models (CGMs), which model the joint distribution $p(x, y)$, would be inherently more robust. The logic was straightforward: a CGM cannot simply ignore "class-irrelevant" information. If an adversary moves an image off the natural "data manifold," the model should assign it a low likelihood, effectively detecting it as an outlier.

This paper investigates whether this promise holds up in practice, specifically testing if CGMs can detect:

  1. Adversarial examples (worst-case outliers).
  2. Ambiguous inputs (average-case outliers).
  3. Incorrectly labeled inputs (in-distribution outliers).

Methods and Experimental Setup

The authors primarily utilize Flow-based models (specifically a modified version of GLOW) because they allow for exact likelihood computation, verifying their findings using PixelCNN++ (an autoregressive model).

To turn these into classifiers, they model the latent space as a Gaussian Mixture Model (GMM), where each class corresponds to a specific Gaussian component. They explore three training variations: standard Maximum Likelihood Estimation (MLE), a reweighted MLE with an additional classification loss, and a novel "Split Prior" approach that partitions the latent space into discriminative and generative subspaces.

They evaluate these models across three datasets:

  • MNIST: The standard benchmark of simple handwritten digits.
  • CIFAR10: A more complex dataset of natural images.
  • BG-MNIST (New): A hybrid dataset where MNIST digits are placed on randomly sampled CIFAR10 backgrounds, isolating the effect of input complexity (entropy) while keeping the classification task simple.

Key Insights: The Theoretical "No-Go" Theorem

The authors prove that even a "near-perfect" generative model can be fooled. Due to the curse of dimensionality, an adversary can find a path to a high-likelihood region of the wrong class without ever passing through a low-likelihood "valley."

Furthermore, there is a fundamental Likelihood-Label Imbalance. In the MLE objective, the "cost" of misclassifying a label (a few bits of information) is dwarfed by the "cost" of modeling thousands of background pixels. Consequently, the model prioritizes getting the pixels right over drawing sharp boundaries between classes.

Key Results: The Interpolation Paradox and Detection Gap

While the models achieved ~99% accuracy on MNIST and successfully detected boundary attacks as "non-images," robustness completely broke down on CIFAR10. Despite reaching ~84% accuracy, 100% of boundary attacks succeeded on CIFAR10 and went completely undetected by the likelihood threshold.

The CIFAR10 "Mashing" Effect

On simple datasets like MNIST, the model correctly assigns low likelihood to interpolations between classes. On CIFAR10, however, the model exhibits an Interpolation Paradox: it often assigns higher likelihood to ambiguous, interpolated images than to real data.

Log-likelihood during interpolation between classes for MNIST (left) and CIFAR10 (right).

This makes it impossible to use likelihood as a reliable out-of-distribution detector for complex images. The BG-MNIST experiment proved that entropy is to blame: adding a complex background to MNIST digits caused the generative model's robustness to break down, even though a simple linear classifier could still identify the digits easily.

The Detection Gap

Another critical failure point is the inability to detect incorrectly labeled in-distribution inputs. The authors showed that the difference in likelihood between a correct class and the most likely "wrong" class is tiny compared to the natural variance of likelihood across the dataset.

Histograms of Negative Log-likelihood for correct vs. top wrong classes on MNIST and CIFAR10.

Because the negative log-likelihood (NLL) distributions for the correct class and the top wrong class overlap almost entirely, class-membership-based outlier detection is fundamentally ineffective.

Conclusion and Future Work

The paper concludes that as long as we rely on standard maximum likelihood estimation, we cannot expect generative models to provide "free" robustness on complex, high-entropy data. The objective simply prioritizes pixel-level modeling over class distinction.

Future directions for the field must include developing new training objectives that better balance the importance of the class label against input features, exploring non-likelihood-based ways to utilize generative models for defense, and investigating new architectural priors to mitigate the "mashing" of class distributions in high dimensions.

Summary updated

Abstract

Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts. They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution. In this work, we investigate robust classification with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises. Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs. Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classifiers. Experimentally, we find that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10. We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective. Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classification.
BibTeX
@inproceedings{fetaya2020understanding,
  author = {Ethan Fetaya and Jörn-Henrik Jacobsen and Will Grathwohl and Richard S. Zemel},
  title = {{Understanding the Limitations of Conditional Generative Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=OChvrX0sFR},
}
Figure