Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?

Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu AizawaSubmission to ICLR 2024 · Withdrawn

TL;DR

When deep neural networks are adapted to downstream tasks via lightweight tuning (e.g., linear probing), they frequently encounter out-of-distribution (OOD) test inputs that overlap with their pre-training data—a setting the authors term PT-OOD (Pre-Trained OOD). Surprisingly, self-supervised models and vision-language backbones like CLIP struggle heavily to detect these familiar OOD inputs compared to supervised models, even when their classification accuracy is higher. The solution? Detecting OOD inputs directly in the pre-trained feature space (e.g., via non-parametric nearest neighbors) bypasses downstream decision boundaries and restores near-perfect detection.

What Problem Does This Paper Address?

Traditional OOD detection research operates under two key assumptions: models are trained from scratch on in-distribution (ID) data, and OOD samples represent completely novel concepts the model has never encountered.

Modern machine learning, however, relies heavily on large pre-trained backbones (trained on millions or billions of web images) adapted to downstream tasks using lightweight methods like linear probing. In real-world deployment, OOD inputs often originate from the same broad data distribution as the pre-training corpus, but do not belong to the downstream ID classes. The authors formulate this realistic threat model as PT-OOD detection. While conventional wisdom holds that stronger ID classifiers make better OOD detectors, this paper demonstrates that representations from popular self-supervised pre-training schemes fail dramatically on PT-OOD data.

Methods

  • Benchmarking Tuning Paradigms: The authors freeze pre-trained vision backbones (CNNs and Vision Transformers) and train a linear classifier (linear probing) on downstream ID datasets.
  • Analysis of Decision Boundaries vs. Feature Spaces: They evaluate both simple baseline detectors (Maximum Softmax Probability, or MSP) and state-of-the-art post-hoc methods (Energy, ReAct, DICE, GradNorm, MaxLogit), alongside multi-layer perceptron (MLP) classification heads.
  • Boundary-Independent Feature-Based Detection: Rather than relying on class-dependent logits and ID decision boundaries, they apply non-parametric $k$-nearest neighbors ($k\text{NN}$) directly on normalized pre-trained feature embeddings: $$S_{k\text{NN}}(x) = - \|z - z_k\|_2$$ where $z$ is the test embedding and $z_k$ is its $k$-th nearest neighbor among ID training features.

  • Hybrid Ensembles: They explore methods like ViM (Virtual-logit Matching) to simultaneously detect PT-OOD and novel (non-PT-OOD) inputs.

Datasets

  • Upstream Pre-training: ImageNet-1K; plus LAION-2B and OpenAI's proprietary web dataset for CLIP/OpenCLIP evaluations.
  • Downstream ID Tasks: CIFAR-10, CIFAR-100, Caltech-101 (excluding human face categories), and Food-101.
  • PT-OOD Benchmarks:
  • ImageNet-30 (IN-30): Filtered to remove semantic overlap with downstream ID classes.
  • ImageNet-20-O (IN-20): A newly curated 20-class subset (940 images) drawn from the ImageNet-1K validation set, verified to avoid overlap with downstream classes.
  • For CLIP: iNaturalist, Places, and SUN.
  • Non-PT-OOD Benchmarks: LSUN-Resize, iSUN, and cross-evaluation between CIFAR-10 and CIFAR-100.

Key Insights and Ideas

  • Transferability Can Compromise Safety: Supervised pre-training enforces tight semantic clusters for each class, which keeps familiar non-target classes tightly separated in feature space. In contrast, self-supervised pre-training encourages broad, generalizable representations. As a result, PT-OOD data is scattered throughout the feature space and easily falls inside downstream ID linear decision boundaries.
  • High Accuracy $\neq$ High OOD Robustness: Self-supervised models like BYOL and SwAV matched or surpassed supervised baselines on downstream classification accuracy, yet suffered catastrophic degradation on PT-OOD detection.
  • Post-Hoc Detectors Fail at the Boundary Level: Standard logit- and gradient-based OOD detectors (Energy, ReAct, GradNorm) cannot fix this vulnerability because the issue is geometric: scattered PT-OOD points are misclassified as ID with high confidence.
  • CLIP Inherits the Same Vulnerability: Because CLIP optimizes for multi-modal alignment across diverse domains, its zero-shot transferability leads to similarly scattered features when evaluated under linear probing.

Key Results

  • Self-Supervised Models Collapse on PT-OOD: Under MSP detection on CIFAR-10 (ResNet-50), supervised pre-training achieved 95.77% AUROC on ImageNet-30, whereas MoCo v2 plummeted to 69.39%. On CIFAR-100 with ViT-S, supervised pre-training achieved 89.54% AUROC, while iBOT and DINO dropped to 55.51% and 56.62% respectively—barely above a random coin flip (50.0%).
  • CLIP Vulnerability: On CIFAR-10, linear-probed CLIP ResNet-50 scored only 61.06% AUROC on ImageNet-30, compared to 95.77% for standard supervised ResNet-50.
  • Feature-Based $k\text{NN}$ Solves PT-OOD: Bypassing the linear head and computing $k\text{NN}$ distances directly on backbone embeddings elevated AUROC across all self-supervised architectures to 99.7%–100.0% (e.g., MoCo v2 on CIFAR-100 jumped from 49.23% to 100.0%).
  • Versatile Detection: For setups mixing both PT-OOD and completely novel OOD data, hybrid methods like ViM (combining feature residuals with logits) performed best, reaching an average AUROC of 95.09%.

Limitations and Future Work

  • Inference Latency: While feature-space $k\text{NN}$ resolves PT-OOD detection, querying nearest neighbors against all stored ID training embeddings introduces high computational and memory overhead during inference compared to single-pass logit scoring.
  • Scope of Adaptation Methods: The paper focuses primarily on linear probing. Parameter-efficient fine-tuning alternatives (e.g., LoRA, Adapters, Visual Prompt Tuning) were not explored.
  • Task Generality: Experiments are limited to image classification; applying the PT-OOD framework to dense tasks (e.g., object detection, segmentation) remains open for future research.

Summary updated

Abstract

Out-of-distribution (OOD) detection is critical for safety-sensitive machine learning applications and has been extensively studied, yielding a plethora of methods developed in the literature. However, most studies for OOD detection did not use pre-trained models and trained a backbone from scratch. In recent years, transferring knowledge from large pre-trained models to downstream tasks by lightweight tuning has become mainstream for training in-distribution (ID) classifiers. To bridge the gap between the practice of OOD detection and current classifiers, the unique and crucial problem is that the samples whose information networks know often come as OOD input. We consider that such data may significantly affect the performance of large pre-trained networks because the discriminability of these OOD data depends on the pre-training algorithm. Here, we define such OOD data as PT-OOD (Pre-Trained OOD) data. In this paper, we aim to reveal the effect of PT-OOD on the OOD detection performance of pre-trained networks from the perspective of pre-training algorithms. To achieve this, we explore the PT-OOD detection performance of supervised and self-supervised pre-training algorithms with linear-probing tuning, the most common efficient tuning method. Through our experiments and analysis, we find that the low linear separability of PT-OOD in the feature space heavily degrades the PT-OOD detection performance, and self-supervised models are more vulnerable to PT-OOD than supervised pre-trained models, even with state-of-the-art detection methods. To solve this vulnerability, we further propose a unique solution to large-scale pre-trained models: Leveraging powerful instance-by-instance discriminative representations of pre-trained models and detecting OOD in the feature space independent of the ID decision boundaries. This study provides significant insights to OOD detection with pre-trained models.
BibTeX
@misc{miyai2023can,
  author = {Atsuyuki Miyai and Qing Yu and Go Irie and Kiyoharu Aizawa},
  title = {{Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?}},
  year = {2023},
  eprint = {2310.00847},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.00847},
}
Figure