Unsupervised Out-of-Domain Detection via Pre-trained Transformers

Keyang Xu, Tongzheng Ren, Shikun Zhang, Yihao Feng, Caiming Xiong

TL;DR

This paper tackles the crucial problem of detecting out-of-domain (OOD) inputs for deployed AI systems, a task that has historically required labeled in-domain data. The authors propose a novel method that leverages the rich, multi-layer representations of pre-trained transformers (like BERT) to build OOD detectors using only unsupervised in-domain data, significantly broadening the applicability of OOD detection.

What Problem Does This Paper Address?

Modern AI systems, especially those deployed in real-world scenarios, are vulnerable to "out-of-domain" (OOD) inputs. These are inputs that differ significantly from the data the model was trained on, and they can lead to unpredictable behavior, incorrect predictions, and even safety issues. Detecting these OOD inputs is vital for building trustworthy AI. However, most existing OOD detection methods rely on having labeled in-domain data to train a classifier. This is a major limitation, as many real-world applications, particularly those not focused on classification (like summarization or topic modeling), lack such labels. Furthermore, the nature of user-generated content means OOD inputs can be diverse and constantly evolving, making it impractical to collect and label them all. This paper addresses this gap by developing an OOD detection method that works without any in-domain labels, using only unlabeled in-domain data.

Methods

The core of the paper's contribution lies in how it extracts and utilizes features from pre-trained transformer models (like BERT and RoBERTa) for OOD detection.

  • Leveraging Multi-Layer Representations: Instead of relying on features from a single layer (e.g., the final [CLS] token embedding), the authors hypothesize that features from all layers of a transformer capture different linguistic properties and are collectively more informative for OOD detection.
  • Mahalanobis Distance Features (MDF): Directly concatenating all layer features is computationally expensive. The authors propose a computationally efficient method to transform these multi-layer features into a lower-dimensional representation using Mahalanobis distance. This approach models the data distribution with hyper-ellipsoids, which are more flexible than the hyperspheres used in traditional one-class SVMs. This MDF representation is then used as input for a standard one-class SVM.
  • Domain-Specific Fine-Tuning: To further boost detection accuracy, two fine-tuning techniques are introduced:
  • In-Domain Masked Language Modeling (IMLM): This involves continuing the masked language modeling pre-training task on the in-domain data, encouraging the model to better capture the nuances of the target domain.
  • Binary Classification with Auxiliary Dataset (BCAD): This method uses the in-domain data as positive samples and a large, unlabeled public dataset (like BooksCorpus or English Wikipedia) as negative samples. A binary classifier is trained on top of the transformer, effectively adapting the model to distinguish between in-domain and general out-of-domain data.

Datasets

The paper evaluates its methods on two distinct datasets:

  • Cross-Corpus Dataset (SST): The in-domain data is the training set of the Stanford Sentiment Treebank (SST). Out-of-domain data is drawn from four other datasets: 20 Newsgroups, English-German Multi30K, RTE, and SNLI. The authors explicitly state they do not use the sentiment labels from SST for training the OOD detector.
  • Cross-Intent Dataset (CLINIC150): This dataset consists of conversational queries. The in-domain data is the training set of CLINIC150, with intent labels discarded. Out-of-domain data comprises queries that do not fall into any of the 150 defined intents.

Key Insights and Ideas

  • No Single Layer is Best: The paper empirically validates that no single layer of a pre-trained transformer consistently provides the best features for OOD detection across different tasks. This strongly motivates the need to leverage information from multiple layers.
  • MDF Outperforms Simpler Methods: The proposed Mahalanobis Distance Features (MDF) method significantly outperforms baselines that use single-layer features, simple aggregations (mean/max pooling), or traditional TF-IDF features. This highlights the power of capturing the distributional shape of multi-layer transformer embeddings.
  • Fine-Tuning is Crucial: Both IMLM and BCAD fine-tuning techniques, when combined with MDF, lead to substantial improvements in OOD detection performance, demonstrating the benefit of adapting pre-trained models to the specific in-domain data distribution.
  • RoBERTa Benefits More from MDF: While BERT's single layers sometimes performed better than RoBERTa's on SST, RoBERTa generally achieved better results when using the MDF approach, suggesting that MDF is particularly effective at extracting robust representations from models that might have different pre-training objectives.

Key Results

  • Significant AUROC Improvements: The MDF method, when applied to RoBERTa, achieved an AUROC of 84.4% on the challenging CLINIC150 dataset, a significant improvement over the best single-layer BERT (59.9%) and TF-IDF (58.5%). For SST, the best MDF-based method (RoBERTa + IMLM + BCAD + MDF) achieved an AUROC of 99.9%.
  • MDF vs. EDF: The MDF approach (using hyper-ellipsoids) consistently outperformed the EDF approach (using hyperspheres, equivalent to Euclidean distance), indicating that the learned features are not invariant across dimensions and benefit from a more flexible distributional model.
  • Fine-tuning Boosts Performance: Combining IMLM and BCAD with MDF yielded the best results, with RoBERTa + IMLM + BCAD + MDF achieving the highest AUROC scores on both datasets.

Limitations and Future Work

  • Computational Cost of MDF: While the MDF approach is computationally efficient for training the one-class SVM (due to the low-dimensional transformed space), the initial extraction of multi-layer features from transformers can still be computationally intensive.
  • Dataset Specificity: The paper notes that the optimal layer for single-layer features is dataset-specific, reinforcing the need for multi-layer approaches.
  • Future Work: The authors express interest in deploying their OOD detection method in real-world applications, such as detecting unseen classes in incremental few-shot learning or filtering OOD samples in data augmentation pipelines.

Summary updated

BibTeX
@inproceedings{xu2021unsupervised,
  author = {Keyang Xu and Tongzheng Ren and Shikun Zhang and Yihao Feng and Caiming Xiong},
  title = {{Unsupervised Out-of-Domain Detection via Pre-trained Transformers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=JMn5MmeDffP},
}
Figure