Out-of-distribution Detection via Frequency-regularized Generative Models

Mu Cai, Yixuan Li

Abstract

Generative models often fail to detect out-of-distribution (OOD) data due to an over-reliance on low-frequency background information rather than semantic object features. We present Frequency-regularized Learning (FRL), a training framework that incorporates high-frequency edge information as an additional input channel. This approach incentivizes models to focus on semantic structures, achieving state-of-the-art OOD detection performance while maintaining inference speeds up to 147x faster than optimization-based alternatives.

Introduction

Deep generative models, such as Variational Autoencoders (VAEs) and Normalizing Flows, are designed to learn the underlying distribution of training data. Ideally, these models should assign high likelihood to in-distribution (ID) data and low likelihood to OOD data. However, a persistent challenge in the field is the "likelihood paradox," where models trained on simple datasets (e.g., MNIST) assign higher likelihoods to complex OOD data (e.g., CIFAR-10).

Research suggests this failure stems from background bias. Generative models tend to overfit to low-frequency information—the smooth, blurry components that typically constitute image backgrounds. If an OOD image shares statistical background similarities with the training set, the model may incorrectly classify it as in-distribution. While previous methods attempted to mitigate this during inference, they are often computationally prohibitive and slow.

Methodology: Frequency-regularized Learning (FRL)

The proposed FRL framework is a "plug-and-play" training mechanism compatible with various generative architectures, including VAEs, GLOW, and PixelCNN++. The core idea is to shift the model's focus from low-frequency backgrounds to high-frequency semantic features.

High-Frequency Extraction

For every training image $x$, a high-frequency component $x_H$ is extracted. This process involves converting the image to grayscale, applying a Gaussian blur to isolate low-frequency information ($x_L$), and subtracting this blur from the original grayscale image. The resulting map captures the contours and fine details—the "skeleton"—of the objects within the image.

The high-frequency component $x_H$ is extracted by subtracting a blurred grayscale version of the image from its original grayscale counterpart.

Model Integration and Training

Rather than processing standard RGB images, the model receives a four-channel input $[R, G, B, x_H]$. By requiring the model to reconstruct or model the likelihood of this high-frequency channel, the training process is regularized to prioritize semantic shapes and object boundaries.

The FRL framework concatenates the high-frequency map $x_H$ with the original RGB image $x$, forming a four-channel input for the generative model.

At test time, OOD detection is performed using a modified likelihood score that integrates both pixel-space and frequency-space information.

Experimental Setup

The researchers evaluated FRL across several benchmarks:

  • Standard Benchmarks: CIFAR-10 and Fashion-MNIST served as ID sets, with SVHN, LSUN, and MNIST as OOD sets.
  • Large-scale Evaluation: CelebA (high-resolution faces) was used as the ID set to test the method's scalability.
  • High-Resolution OOD Sets: For CelebA tasks, OOD data included iNaturalist, Places, SUN, and Textures, all resized to 128x128.

Key Results

FRL demonstrates significant improvements in both accuracy and efficiency:

  • Superior Detection Accuracy: On the CelebA task, FRL outperformed the competitive "Likelihood Regret" baseline by 10.7% in AUROC.
  • Computational Efficiency: Unlike methods that require iterative optimization during inference, FRL is a training-time fix. This results in a 147x speedup over Likelihood Regret, enabling real-time OOD detection at approximately 169 images per second.
  • Preserved Generative Quality: The addition of the frequency channel does not degrade reconstruction performance. Metrics such as PSNR and SSIM showed slight improvements, indicating that the model retains its generative capabilities while becoming more robust.

Conclusion and Future Work

By emphasizing high-frequency components, FRL effectively mitigates background bias in generative models. This simple architectural change—adding a fourth input channel—provides a robust and efficient solution for OOD detection.

Future research may explore the application of FRL to non-visual domains, such as audio or time-series data, where frequency analysis is equally relevant. Additionally, investigating more sophisticated filtering techniques, such as Wavelets or Fourier transforms, could further enhance the model's ability to distinguish between semantic content and background noise.

Summary updated

BibTeX
@inproceedings{cai2023out,
  author = {Mu Cai and Yixuan Li},
  title = {{Out-of-distribution Detection via Frequency-regularized Generative Models}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00548},
}
Figure