Out-of-Distribution Detection with a Single Unconditional Diffusion Model

Alvin Heng, Alexandre H. Thiery, Harold Soh

TL;DR

Traditional generative out-of-distribution (OOD) detection requires training a separate model for every single in-distribution dataset. This paper introduces Diffusion Paths (DiffPath), a novel framework that leverages a single pretrained unconditional diffusion model to perform OOD detection across multiple diverse tasks. By analyzing the geometric properties—specifically the rate-of-change and curvature—of the trajectories that samples take during diffusion inversion, DiffPath achieves competitive performance without model retraining.

What Problem Does This Paper Address?

Deep neural networks are notoriously overconfident when presented with out-of-distribution (OOD) data, which poses severe risks in safety-critical deployments like robotics or healthcare. To detect these anomalies, unsupervised methods typically train a generative model on the "in-distribution" (inlier) data. If a test sample has a low likelihood under the model, it's flagged as OOD.

However, this paradigm has two major flaws:

  1. The Retraining Burden: You must train and maintain a completely new generative model for every new inlier dataset or task.
  2. The Likelihood Paradox: Deep generative models often assign higher likelihoods to OOD samples than to inlier data (e.g., a model trained on CIFAR-10 might find simpler SVHN digits more "likely" than its own training images).

The authors ask a fundamental question: Can we use a single, pretrained generative foundation model to perform OOD detection across entirely different datasets?

Methods

Instead of relying on flawed likelihood metrics, the authors introduce DiffPath. The core idea is to take a single unconditional diffusion model (pretrained on a diverse dataset like ImageNet or CelebA) and use it to propagate test samples backward along the diffusion trajectory toward a standard normal distribution using DDIM inversion.

Figure 1: Illustration of the diffusion paths of sam- ples from two different distributions (CIFAR10 and SVHN) obtained via DDIM integration. The paths have different first and second derivatives (rate-of- change and curvature). We propose to measure these quantities for OOD detection.

As shown in Figure 1, different data distributions follow distinct geometric trajectories. DiffPath quantifies these differences by measuring two key properties of the path:

  • Rate-of-Change (First-Order): Measured by the $L_2$ norm of the predicted score function $\lVert \epsilon_\theta(x_t, t) \rVert_2$.
  • Curvature (Second-Order): Measured by the time derivative of the score function $\lVert \partial_t \epsilon_\theta(x_t, t) \rVert_2$. To avoid computationally expensive Jacobian-Vector Products (JVPs), the authors approximate this using a simple, cost-free finite difference between consecutive DDIM steps.

The authors propose two variants:

  • DiffPath-1D: Uses a one-dimensional curvature statistic, fitting a Kernel Density Estimator (KDE) to the inlier training set's statistics.
  • DiffPath-6D: A higher-dimensional variant that concatenates the first, second, and third powers of both the score and its derivative. This preserves sign information, preventing edge cases like an image and its negative from yielding identical statistics. A Gaussian Mixture Model (GMM) is used for scoring here.

Datasets

The method was evaluated using diffusion models pretrained on:

  • ImageNet ($64 \times 64$)
  • CelebA ($32 \times 32$)

The OOD detection capabilities were tested across various pairwise combinations of standard benchmarks acting as inliers and outliers: CIFAR-10 (C10), SVHN, CelebA, CIFAR-100 (C100), and Textures.

Key Insights and Ideas

  • The Power of Foundation Models: A single diffusion model trained on a sufficiently broad dataset (like ImageNet) can successfully invert completely unseen distributions into standard normal noise.
  • Geometry Over Likelihood: While the likelihoods of different datasets overlap confusingly, the geometry of their trajectories does not. The diffusion path closely mirrors the Optimal Transport path; hence, the velocity and bending of these paths act as highly sensitive statistical signatures for a dataset.
Figure 2: Histograms of various statistics of the respective training sets. The NLL is calculated using a diffusion model trained on CIFAR10, while the other two statistics are calculated with a model trained on ImageNet.
  • Curvature Matters: As seen in Figure 2, moving from a first-order score norm to a second-order curvature statistic significantly sharpens the separation between inliers and outliers, boosting the AUROC from 0.856 to 0.965 on CIFAR-10 vs SVHN.

Key Results

  • Single Model Efficiency: DiffPath-6D-CelebA achieved an average AUROC of 0.931 across all evaluation tasks. This matches or exceeds the performance of state-of-the-art baselines (like MSMA) that require separate, dedicated models for each dataset.
  • Zero-Shot OOD Detection: Even when using the ImageNet model—which never saw any of the evaluation datasets during training—DiffPath achieved a highly competitive average AUROC of 0.850.
  • Massive Speedup: DiffPath requires only 10 Number of Function Evaluations (NFEs). This makes it orders of magnitude faster than reconstruction-based diffusion baselines like LMD or DDPM-OOD, which require hundreds or thousands of evaluations due to iterative reconstruction loops.

Limitations and Future Work

  • Near-OOD Performance: Like most generative OOD detection methods, DiffPath struggles with "Near-OOD" tasks where distributions heavily overlap semantically (such as distinguishing CIFAR-10 from CIFAR-100). Discriminative methods still hold the upper hand here.
  • Approximated Curvature: The authors only compute the simple time derivative component of the path curvature to save on compute. Calculating the full analytical derivative (including the JVP terms) might yield even better accuracy at the expense of inference speed.
  • Domain Constraints: The models evaluated were trained on natural images. Applying DiffPath to specialized domains, such as medical imaging or completely different modalities (audio, text, tabular data), remains an open avenue for future research.

Summary updated

Abstract

Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. However, such approaches require a new model to be trained for each inlier dataset. This paper explores whether a single model can perform OOD detection across diverse tasks. To that end, we introduce Diffusion Paths (DiffPath), which uses a single diffusion model originally trained to perform unconditional generation for OOD detection. We introduce a novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal. Extensive experiments show that with a single model, DiffPath is competitive with prior work using individual models on a variety of OOD tasks involving different distributions. Our code is publicly available at https://github.com/clear-nus/diffpath.
BibTeX
@inproceedings{heng2024out,
  author = {Alvin Heng and Alexandre H. Thiery and Harold Soh},
  title = {{Out-of-Distribution Detection with a Single Unconditional Diffusion Model}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=tTnFH7D1h4},
}
Figure