Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks

Mateusz Baran, Joanna Baran, Mateusz Wójcik, Maciej Zieba, Adam Gonczarek

TL;DR

This paper benchmarks eight popular out-of-distribution (OOD) detection methods for text classification tasks, revealing that current methods struggle with subtle distributional shifts like shuffled word order or background changes. The authors provide a reproducible framework and highlight the need for more robust OOD detection techniques in NLP to ensure AI safety and trustworthiness.

What Problem Does This Paper Address?

Modern AI systems, especially in Natural Language Processing (NLP), are often trained in controlled environments (in-distribution data). However, when deployed in the real world, they encounter data that deviates from this training distribution (out-of-distribution or OOD data). This is problematic because models can become overconfident in their incorrect predictions on OOD data, leading to errors and a lack of trustworthiness. While OOD detection methods exist, their effectiveness in NLP, particularly for nuanced shifts beyond simple domain changes, is not well understood. This paper aims to bridge this gap by comprehensively evaluating existing OOD detection techniques within the NLP context, focusing on methods that are easy to integrate into existing systems.

Methods

The authors focus on "post-hoc" OOD detection methods, meaning they can be applied to already trained models without requiring additional training data or architectural changes. They reimplemented and evaluated eight such methods:

  • Maximum Softmax Probability (MSP): A baseline method using the softmax output of the model.
  • Energy-based: Uses an "energy score" to gauge model confidence.
  • Rectified Activations (ReAct): Truncates high activations to reduce overconfidence.
  • KL-Matching (KLM): Compares model output probabilities to class-conditional distributions.
  • GradNorm: Analyzes gradients from the classification layer.
  • Directed Sparsification (DICE): Selectively sparsifies model weights.
  • Virtual-logit Matching (ViM): Combines feature and output logit information.
  • K-nearest neighbors (KNN): Compares input embeddings to training set embeddings.

These methods were integrated into a fine-tuned RoBERTa model for text classification.

Datasets

The study utilizes several datasets for evaluation, categorized into two main scenarios:

Scenario 1: Similarity-based Splits This scenario categorizes OOD data based on its perceived similarity to the in-distribution (ID) data.

  • News Category (NC/I): Used as the primary in-distribution dataset.
  • News Category (NC/O): A near-OOD split with semantically similar but distinct classes.
  • Twitter Topic Classification: Another near-OOD dataset with similar topics but different sentence structures.
  • SST-2, Yelp, IMDB: Far-OOD datasets (movie reviews, etc.) representing a larger domain shift.
  • Language Detection Dataset: A distinct-OOD dataset, specifically using non-English texts to challenge the English-trained RoBERTa model.
  • News Category Random: A novel benchmark created by randomly shuffling words within in-domain texts to simulate severe structural disruption.

Scenario 2: Semantic and Background Shift This scenario focuses on specific types of distribution shifts.

  • 20 Newsgroups: Used to create semantic shifts by training on subsets of categories (e.g., Computer, Sports, Politics) and testing on others.
  • IMDB, SST-2, Yelp: Used to test background shifts, where datasets have similar linguistic properties but different topics.

Key Insights and Ideas

  • Current OOD methods struggle with subtle shifts: The most significant finding is that existing OOD detection methods are not sensitive enough to reliably detect samples with background shifts or randomly shuffled word order, even within the same domain.
  • KNN and ViM emerge as strong performers: Among the tested methods, KNN and ViM consistently showed the best performance across various scenarios, outperforming the baseline MSP.
  • Language shift is relatively easy to detect: Models and OOD detectors could easily identify texts in different languages as OOD, likely due to the distinct token distributions.
  • Shuffled in-domain text is a major challenge: The "News Category Random" benchmark revealed that while models might struggle with semantically distinct OOD data, they can also fail to identify structurally disrupted in-domain data as OOD, which is a critical failure for AI safety.
  • Background shift is particularly difficult: Methods had a harder time distinguishing OOD samples when the shift was related to background characteristics (like writing style or topic) rather than clear semantic class changes.

Key Results

  • KNN and ViM consistently achieved higher AUROC scores compared to other methods across most Near-OOD, Far-OOD, and Distinct-OOD scenarios. For instance, KNN achieved AUROC scores as high as 99.8% on the Twitter dataset, while ViM also performed strongly.
  • DICE and KLM were identified as the worst-performing approaches in many NLP tasks, with measures sometimes close to random chance (around 50%).
  • In Scenario 2, ViM was more effective for semantic shifts, while KNN outperformed ViM on background shifts for IMDB and Yelp datasets.
  • The "News Category Random" dataset significantly increased model confidence in detecting OOD when compared to the original NC/O dataset, but the OOD methods themselves struggled to identify these shuffled in-domain texts as OOD.

Limitations and Future Work

  • Limited to text classification: The study was confined to text classification and did not explore other NLP tasks like Named Entity Recognition (NER) or Question Answering (QA). Expanding to these areas could offer a broader understanding of method performance.
  • Single model architecture: Experiments were conducted using only the RoBERTa model. Testing with other architectures, especially larger language models (LLMs), could reveal different performance characteristics.
  • No hyperparameter search: Due to computational constraints, hyperparameters for the OOD methods were set to recommended values from original papers, which might have impacted the results. Future work could involve a thorough hyperparameter search.
  • Need for more robust methods: The authors emphasize the need for future research to develop more effective OOD detection approaches for NLP that can better handle semantic disorders and structural disruptions in input sentences. The current low overall performance, especially on Near-OOD samples, indicates significant room for improvement.

Summary updated

Abstract

State-of-the-art models can perform well in controlled environments, but they often struggle when presented with out-of-distribution (OOD) examples, making OOD detection a critical component of NLP systems. In this paper, we focus on highlighting the limitations of existing approaches to OOD detection in NLP. Specifically, we evaluated eight OOD detection methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications. One of our contributions is providing a well-structured research environment that allows for full reproducibility of the results. Additionally, our analysis shows that existing OOD detection methods for NLP tasks are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts. Particularly challenging testing scenarios arise in cases of background shift and randomly shuffled word order within in domain texts. This highlights the need for future work to develop more effective OOD detection approaches for the NLP problems, and our work provides a well-defined foundation for further research in this area.
BibTeX
@inproceedings{baran2023classical,
  author = {Mateusz Baran and Joanna Baran and Mateusz Wójcik and Maciej Zieba and Adam Gonczarek},
  title = {{Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=l97KlDmCSW},
}
Figure