Out-of-distribution Detection via Frequency-regularized Generative Models

Mu Cai, Yixuan Li

Introduction

Deep generative models (DGMs), such as Variational Autoencoders (VAEs) and Flow-based models, are designed to learn the underlying distribution of data. Ideally, these models should assign high likelihood to samples from the training distribution (In-Distribution, or ID) and low likelihood to samples from outside that distribution (Out-of-Distribution, or OOD). However, a persistent "likelihood paradox" has been observed: DGMs often assign higher likelihoods to OOD data than to ID data. For instance, a model trained on the complex CIFAR-10 dataset might assign a higher likelihood to simple MNIST digits.

Recent research suggests this failure stems from an over-reliance on low-frequency background information. Because backgrounds are often similar across different datasets, models "overfit" to these commonalities rather than learning the unique semantic features of the target objects. This paper introduces Frequency-regularized Learning (FRL), a framework designed to shift the model's focus toward high-frequency semantic components like contours and textures.

Frequency-regularized Learning (FRL)

The core of the FRL framework is the extraction and utilization of high-frequency image components. By forcing the model to reconstruct these components, it is incentivized to prioritize structural information over generic background pixels.

High-Frequency Extraction

To obtain the high-frequency component $x_H$ from an input image $x$, the authors employ a simple yet effective preprocessing pipeline involving grayscale transformation and Gaussian blurring to isolate the high-frequency details.

The process of extracting the high-frequency component $x_H$ from an input image $x$ via Gaussian blur subtraction.

Input Augmentation and Training

Rather than replacing the original image, the high-frequency component $x_H$ is concatenated to the original RGB image as a fourth input channel. The resulting augmented input $x_F = [x, x_H]$ is then fed into the generative model. This "plug-and-play" approach requires minimal architectural changes and is compatible with various DGM families, including VAEs, GLOW, and PixelCNN++.

Overview of the FRL framework integrated into a Variational Autoencoder (VAE) architecture, showing the reconstruction of both pixel space and high-frequency inputs.

Experimental Evaluation

The authors evaluated FRL across a wide range of benchmarks. In-distribution datasets included CIFAR-10, Fashion-MNIST, and CelebA. OOD datasets spanned SVHN, MNIST, and large-scale sets like iNaturalist and SUN.

Key Results

  • Detection Accuracy: On the CelebA dataset, FRL achieved a 10.7% improvement in AUROC compared to the "Likelihood Regret" baseline, a previous state-of-the-art method.
  • Inference Efficiency: Unlike methods that require expensive optimization at test time, FRL relies on a standard forward pass. This results in a 147× speedup over Likelihood Regret on high-resolution images.
  • Robustness: The method proved effective across different architectures, demonstrating that frequency regularization addresses a fundamental flaw in how DGMs learn, rather than a specific architectural weakness.

Conclusion

Frequency-regularized Learning (FRL) offers a computationally efficient and architecturally agnostic solution to the likelihood paradox in OOD detection. By guiding generative models to focus on high-frequency semantic features, FRL significantly improves detection accuracy while maintaining the speed necessary for real-world deployment. Future work may explore the application of these principles to non-visual modalities such as audio or time-series data.

Summary updated

Abstract

Modern deep generative models can assign high likelihood to inputs drawn from outside the training distribution, posing threats to models in open-world deployments. While much research attention has been placed on defining new test-time measures of OOD uncertainty, these methods do not fundamentally change how deep generative models are regularized and optimized in training. In particular, generative models are shown to overly rely on the background information to estimate the likelihood. To address the issue, we propose a novel frequency-regularized learning (FRL) framework for OOD detection, which incorporates high-frequency information into training and guides the model to focus on semantically relevant features. FRL effectively improves performance on a wide range of generative architectures, including variational auto-encoder, GLOW, and PixelCNN++. On a new large-scale evaluation task, FRL achieves the state-of-the-art performance, outperforming a strong baseline Likelihood Regret by 10.7% (AUROC) while achieving 147× faster inference speed. Extensive ablations show that FRL improves the OOD detection performance while preserving the image generation quality. Code is available at https://github.com/mu-cai/FRL.
BibTeX
@inproceedings{cai2023out,
  author = {Mu Cai and Yixuan Li},
  title = {{Out-of-distribution Detection via Frequency-regularized Generative Models}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00548},
}
Figure