Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?

Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu Aizawa

Introduction: The Emergence of PT-OOD Data

As machine learning shifts from training models from scratch to fine-tuning massive pre-trained backbones, a new security flaw has been identified: PT-OOD (Pre-Trained Out-of-Distribution) data. Traditional OOD detection research assumes models encounter entirely novel data. However, in transfer learning, a model may encounter samples that were part of its massive pre-training set but are irrelevant to its specific downstream task.

Conceptual overview of PT-OOD data relative to In-Distribution (ID) and non-PT-OOD data.

For example, a model pre-trained on ImageNet but fine-tuned only for "Car" recognition might encounter a "Toaster." While the model "knows" what a toaster is from pre-training, it is technically OOD for the car-recognition task. The researchers found that models—particularly self-supervised ones—often mistake these "familiar outsiders" for in-distribution classes with dangerously high confidence.

Methodology

The study systematically compared two pre-training paradigms:

  1. Supervised Pre-training: Models trained with explicit labels (e.g., standard ImageNet training).
  2. Self-supervised Pre-training: Models like BYOL, SwAV, MoCo v2, iBOT, and DINO that learn representations from unlabeled data.

The researchers used linear probing to adapt these backbones to downstream tasks (CIFAR-10/100, Caltech-101, Food-101). They evaluated detection performance using standard Maximum Softmax Probability (MSP) and a proposed k-Nearest Neighbors (kNN) approach in the feature space.

Key Insights: The "Transferability" Trap

A core finding is that the very feature that makes self-supervised learning (SSL) powerful—its ability to learn highly "transferable" general representations—makes it vulnerable to PT-OOD data.

Comparison of decision boundaries for supervised vs. self-supervised pre-training.

In supervised pre-training, classes are forced into tight, distinct clusters. This keeps PT-OOD data linearly separable in the feature space. In contrast, SSL models attempt to map the entire data manifold into a useful space. Consequently, PT-OOD samples often fall directly within the decision boundaries of downstream ID classes. This leads to "deceptive confidence," where the model is more certain about a familiar OOD sample than a genuine ID sample.

Key Results and the kNN Solution

The performance gap is stark. On a CIFAR-10 task using a ResNet-50 backbone:

  • Supervised models achieved a PT-OOD detection AUROC of 95.77%.
  • Self-supervised models (MoCo v2) plummeted to 69.39%.

Even foundation models like CLIP struggled, showing significantly lower AUROC than supervised baselines when detecting familiar ImageNet samples (e.g., ~61% vs ~95% AUROC).

However, the researchers discovered a robust fix. By bypassing the classifier's decision boundaries and using kNN-based detection in the feature space, AUROC scores for PT-OOD detection jumped to nearly 100% across almost all methods. This indicates that the model's features can distinguish OOD data; the failure lies in how the final classification layer interprets those features.

Limitations and Future Directions

While the kNN fix is effective, it introduces higher computational costs during inference. Future work is needed to develop more efficient feature-based detectors. Additionally, the study focused on image classification; the impact of PT-OOD data on more complex tasks like object detection or segmentation remains an open question. Finally, exploring how full fine-tuning or parameter-efficient methods (like LoRA) affect this vulnerability could provide further safety insights.

Summary updated

Abstract

Out-of-distribution (OOD) detection is critical for safety-sensitive machine learning applications and has been extensively studied, yielding a plethora of methods developed in the literature. However, most studies for OOD detection did not use pre-trained models and trained a backbone from scratch. In recent years, transferring knowledge from large pre-trained models to downstream tasks by lightweight tuning has become mainstream for training in-distribution (ID) classifiers. To bridge the gap between the practice of OOD detection and current classifiers, the unique and crucial problem is that the samples whose information networks know often come as OOD input. We consider that such data may significantly affect the performance of large pre-trained networks because the discriminability of these OOD data depends on the pre-training algorithm. Here, we define such OOD data as PT-OOD (Pre-Trained OOD) data. In this paper, we aim to reveal the effect of PT-OOD on the OOD detection performance of pre-trained networks from the perspective of pre-training algorithms. To achieve this, we explore the PT-OOD detection performance of supervised and self-supervised pre-training algorithms with linear-probing tuning, the most common efficient tuning method. Through our experiments and analysis, we find that the low linear separability of PT-OOD in the feature space heavily degrades the PT-OOD detection performance, and self-supervised models are more vulnerable to PT-OOD than supervised pre-trained models, even with state-of-the-art detection methods. To solve this vulnerability, we further propose a unique solution to large-scale pre-trained models: Leveraging powerful instance-by-instance discriminative representations of pre-trained models and detecting OOD in the feature space independent of the ID decision boundaries. This study provides significant insights to OOD detection with pre-trained models.
BibTeX
@misc{miyai2023can,
  author = {Atsuyuki Miyai and Qing Yu and Go Irie and Kiyoharu Aizawa},
  title = {{Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?}},
  year = {2023},
  eprint = {2310.00847},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.00847},
}
Figure