While researchers hoped that class-conditional generative models would be naturally robust to adversarial attacks by modeling the entire input distribution, recent findings prove otherwise. This paper demonstrates a theoretical impossibility result for guaranteed robustness and shows that while these models work for simple data like MNIST, they fail catastrophically on complex datasets like CIFAR10 due to the fundamental nature of the maximum likelihood objective.
The Promise of Conditional Generative Models
Deep learning has a well-documented "fragility" problem: discriminative classifiers, which only model the conditional probability $p(y|x)$, are easily fooled by tiny, adversarial perturbations. A popular hypothesis was that Conditional Generative Models (CGMs), which model the joint distribution $p(x, y)$, would be inherently more robust. The logic was straightforward: a CGM cannot simply ignore "class-irrelevant" information. If an adversary moves an image off the natural "data manifold," the model should assign it a low likelihood, effectively detecting it as an outlier.
This paper investigates whether this promise holds up in practice, specifically testing if CGMs can detect:
- Adversarial examples (worst-case outliers).
- Ambiguous inputs (average-case outliers).
- Incorrectly labeled inputs (in-distribution outliers).
Methods and Experimental Setup
The authors primarily utilize Flow-based models (specifically a modified version of GLOW) because they allow for exact likelihood computation, verifying their findings using PixelCNN++ (an autoregressive model).
To turn these into classifiers, they model the latent space as a Gaussian Mixture Model (GMM), where each class corresponds to a specific Gaussian component. They explore three training variations: standard Maximum Likelihood Estimation (MLE), a reweighted MLE with an additional classification loss, and a novel "Split Prior" approach that partitions the latent space into discriminative and generative subspaces.
They evaluate these models across three datasets:
- MNIST: The standard benchmark of simple handwritten digits.
- CIFAR10: A more complex dataset of natural images.
- BG-MNIST (New): A hybrid dataset where MNIST digits are placed on randomly sampled CIFAR10 backgrounds, isolating the effect of input complexity (entropy) while keeping the classification task simple.
Key Insights: The Theoretical "No-Go" Theorem
The authors prove that even a "near-perfect" generative model can be fooled. Due to the curse of dimensionality, an adversary can find a path to a high-likelihood region of the wrong class without ever passing through a low-likelihood "valley."
Furthermore, there is a fundamental Likelihood-Label Imbalance. In the MLE objective, the "cost" of misclassifying a label (a few bits of information) is dwarfed by the "cost" of modeling thousands of background pixels. Consequently, the model prioritizes getting the pixels right over drawing sharp boundaries between classes.
Key Results: The Interpolation Paradox and Detection Gap
While the models achieved ~99% accuracy on MNIST and successfully detected boundary attacks as "non-images," robustness completely broke down on CIFAR10. Despite reaching ~84% accuracy, 100% of boundary attacks succeeded on CIFAR10 and went completely undetected by the likelihood threshold.
The CIFAR10 "Mashing" Effect
On simple datasets like MNIST, the model correctly assigns low likelihood to interpolations between classes. On CIFAR10, however, the model exhibits an Interpolation Paradox: it often assigns higher likelihood to ambiguous, interpolated images than to real data.
This makes it impossible to use likelihood as a reliable out-of-distribution detector for complex images. The BG-MNIST experiment proved that entropy is to blame: adding a complex background to MNIST digits caused the generative model's robustness to break down, even though a simple linear classifier could still identify the digits easily.
The Detection Gap
Another critical failure point is the inability to detect incorrectly labeled in-distribution inputs. The authors showed that the difference in likelihood between a correct class and the most likely "wrong" class is tiny compared to the natural variance of likelihood across the dataset.
Because the negative log-likelihood (NLL) distributions for the correct class and the top wrong class overlap almost entirely, class-membership-based outlier detection is fundamentally ineffective.
Conclusion and Future Work
The paper concludes that as long as we rely on standard maximum likelihood estimation, we cannot expect generative models to provide "free" robustness on complex, high-entropy data. The objective simply prioritizes pixel-level modeling over class distinction.
Future directions for the field must include developing new training objectives that better balance the importance of the class label against input features, exploring non-likelihood-based ways to utilize generative models for defense, and investigating new architectural priors to mitigate the "mashing" of class distributions in high dimensions.
Summary updated