Estimating Semantic Similarity between In-Domain and Out-of-Domain Samples

Rhitabrat Pokharel, Ameeta AgrawalACL 2023

TL;DR

This paper explores how to identify "out-of-domain" (OOD) samples—data that differs from what a machine learning model was trained on—without needing a trained model. They found that Wasserstein distance, a measure of how "far apart" two probability distributions are, shows promising correlation with model performance differences, suggesting it could be a useful unsupervised OOD detection metric.

What Problem Does This Paper Address?

Machine learning models are often deployed in real-world scenarios where the data they encounter might not perfectly match the data they were trained on. This "out-of-domain" (OOD) or "out-of-distribution" (OODist) data can lead to decreased model performance and potentially significant consequences, especially in critical domains like healthcare or law. While previous research has explored OOD detection, it often relies on supervised methods or trained models, which aren't always available. This paper aims to bridge this gap by investigating unsupervised methods to identify OOD samples, focusing on how semantic similarity between training and test data relates to model performance.

Methods

The authors tackle this problem in two main phases:

  1. Performance-based analysis: They first establish a baseline by analyzing how a model's performance (accuracy or F1 score) changes when tested on in-domain (ID) versus OOD data. This phase confirms that performance degradation is a common indicator of OOD samples, but it requires a supervised model.
  2. Unsupervised similarity metrics: To overcome the limitation of needing a supervised model, they explore several unsupervised metrics to estimate the semantic similarity (or distance) between training and test datasets. The core hypothesis is that if an unsupervised metric strongly correlates with the performance drop observed in OOD settings, it can serve as a reliable indicator for OOD detection.

They specifically investigate four similarity metrics:

  • Cosine Similarity: Measures the similarity between sample embeddings.
  • Mauve Score: A more advanced metric for comparing text distributions.
  • Wasserstein Distance (Wstn): Measures the "earth mover's distance" between probability distributions, indicating how much "work" is needed to transform one into the other.
  • Jensen Shannon Distance (JSD): Quantifies the similarity between probability distributions.

To understand the relationship between performance and similarity, they use Kendall Tau and Pearson correlation coefficients.

Datasets

The study utilizes a diverse set of datasets across four common NLP tasks:

  • Sentiment Analysis: IMDb, SST2, Yelp
  • Multiple Choice Question Answering (MCQ): SCIQ, CommonsenseQA (CS), QASC
  • Extractive Question Answering (QA): SQuAD, News, Trivia (from the MRQA dataset)
  • Natural Language Inference (NLI): MNLI, QNLI, WNLI (from the GLUE benchmark)

A key data preparation step involved downsampling datasets to ensure comparable sizes across tasks and balancing class distributions where possible, addressing a common oversight in prior work.

Key Insights and Ideas

  • Performance as an OOD Indicator: The study confirms that model performance (accuracy/F1) generally decreases on OOD samples compared to ID samples. However, this isn't universally true, and the paper notes exceptions, highlighting the need for more robust detection methods.
  • Wasserstein Distance Shines: Among the unsupervised similarity metrics, Wasserstein distance (Wstn) consistently showed the strongest correlation with performance differences across various datasets and tasks, particularly when using Kendall Tau correlation. This suggests it's a promising candidate for unsupervised OOD detection.
  • JSD is Less Informative: Jensen Shannon Distance (JSD) consistently exhibited the weakest correlation with performance, making it less suitable for this task.
  • The Nuance of OOD/OODist: The paper highlights that the terms "OOD" and "OODist" are used inconsistently in prior literature, sometimes referring to different datasets for the same task, and other times to subsets of the same dataset or datasets collected at different times. This ambiguity underscores the need for clear definitions and systematic analysis.

Key Results

The paper reports that Wasserstein distance demonstrated the most consistent correlation with performance differences across 10 out of 12 datasets when using Kendall Tau. Pearson correlation also found Wasserstein and Cosine similarity to be acceptable metrics in 9 out of 12 cases. For instance, in Table 3, we see that for the IMDb dataset, when trained on IMDb and tested on Yelp (an OOD setting), the performance drops from 0.90 to 0.87, and the Wasserstein distance between IMDb and Yelp is relatively low, indicating a correlation between performance drop and similarity. Conversely, when tested on SST2 (a more distant OOD setting), performance drops to 0.17, and the similarity metrics would reflect a larger distance.

Limitations and Future Work

The authors acknowledge several limitations:

  • ID Performance < OOD Performance: In some cases, ID performance was observed to be less than OOD performance. Further investigation is needed to understand this phenomenon.
  • English-Centric: The entire analysis is based on English language datasets. Future work should explore generalization to other languages.

Promising future directions include:

  • Threshold Determination: Determining appropriate thresholds for unsupervised metrics to definitively classify a sample as OOD.
  • Embedding Robustness: Investigating the robustness of these similarity metrics when estimated using different types of embeddings.

Summary updated

BibTeX
@inproceedings{pokharel2023estimating,
  author = {Rhitabrat Pokharel and Ameeta Agrawal},
  title = {{Estimating Semantic Similarity between In-Domain and Out-of-Domain Samples}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure