Out-of-Distribution Detection with Distance Guarantee in Deep Generative Models

Yufeng Zhang, Wanwei Liu, Zhenbang Chen, Ji Wang, Zhiming Liu, Kenli Li, Hongmei Wei, Zuoning Chen

Introduction: The Likelihood Paradox

Deep generative models (DGMs), such as flow-based models and Variational Autoencoders (VAEs), are designed to learn the underlying distribution of training data. Ideally, these models should assign high likelihoods to in-distribution (ID) data and low likelihoods to out-of-distribution (OOD) data. However, a significant "likelihood paradox" has been observed: models trained on complex datasets like CIFAR-10 often assign higher likelihoods to simpler OOD datasets like SVHN. This failure makes raw likelihood an unreliable metric for anomaly detection and leaves models vulnerable to simple data manipulations like contrast adjustments.

Theoretical Foundation: KL Divergence vs. Likelihood

The core of the issue lies in how OOD data is mapped into the latent space. During training, the model attempts to map ID data to a simple prior distribution (e.g., a standard Gaussian). Paradoxically, OOD data often gets "pushed" into high-probability regions of this prior, leading to high likelihood scores.

Theoretical framework showing how KL divergence preserves OOD signals in latent space even when likelihoods are high.

As illustrated in Figure 1, while the likelihood $p(x)$ might be misleading, the Kullback-Leibler (KL) divergence between the distribution of OOD representations ($q_z$) and the prior ($p_z^r$) remains large. The authors prove that a diffeomorphism (the mapping $f(x)$) preserves KL divergence. Thus, if ID and OOD data are well-separated in the data space, they must remain separated in the latent space via KL divergence, even if their point-wise likelihoods overlap.

Methodology: The KLODS Approach

To leverage this theoretical insight, the authors propose KLODS (KL divergence-based OOD detection with Split representations). The method focuses on three key technical innovations:

  1. Last-Scale Focus: In multi-scale flow models, the "last scale" captures high-level, dataset-specific features. OOD data typically deviates most significantly from the training distribution at this final stage.
  2. Vertical Splitting: To enable detection for small batches or even single images (Point-wise Anomaly Detection), the latent representation is split "vertically." By grouping dimensions at the same spatial position, the model captures local pixel dependencies that characterize ID data but are disrupted in OOD samples.
  3. Analytical KL Estimation: KLODS uses fitted Gaussians to analytically calculate a lower bound for the KL divergence, ensuring robust detection even when OOD distributions are non-Gaussian.
Decomposition of KL divergence and the final KLODS criterion using split representations.

As shown in Figure 2, the total KL divergence can be decomposed. The final criterion, $D_g[q_{z3}] > D_g[p_{z3}]$, effectively separates OOD representations ($q_z$) from ID representations ($p_z$) by focusing on the specific components where they diverge most from the prior.

Experimental Results and Robustness

KLODS was evaluated across standard benchmarks (e.g., CIFAR-10 vs. SVHN) and synthetic noise datasets. It demonstrated state-of-the-art performance in two primary tasks:

  • Group Anomaly Detection (GAD): Achieved an average 98.1% AUROC with a batch size of only 5, significantly outperforming previous methods.
  • Point-wise Anomaly Detection (PAD): Achieved 90.7% AUROC, proving effective for individual image assessment.

Crucially, KLODS exhibited remarkable robustness. While existing methods like the "Typicality Test" failed completely (0% AUROC) when OOD data was subjected to contrast manipulation, KLODS maintained its performance, suggesting it captures more fundamental structural differences rather than superficial statistics.

Conclusion

By shifting the focus from raw likelihood to KL divergence in the latent space, KLODS provides a mathematically grounded solution to the likelihood paradox. While its performance is still bounded by the quality of the underlying generative model, it represents a significant step toward reliable and robust OOD detection in deep learning systems.

Summary updated

BibTeX
@article{zhang2024out,
  author = {Yufeng Zhang and Wanwei Liu and Zhenbang Chen and Ji Wang and Zhiming Liu and Kenli Li and Hongmei Wei and Zuoning Chen},
  title = {{Out-of-Distribution Detection with Distance Guarantee in Deep Generative Models}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2024},
  doi = {10.1109/tkde.2023.3309853},
}
Figure