Reliable AI through Out-of-Distribution Detection: A Benchmark Study

José Lucas De Melo Costa, Eliott BarbotOpenReview 2023 · OpenReview

Abstract

This article delves into the issue of detecting out-of-distribution (OOD) examples in machine learning models, with a focus on natural language processing (NLP) applications. It is crucial to identify OOD data to build reliable AI, as our models are not trained to handle such examples and may perform poorly on them. To tackle this problem, we implement and compare various OOD detection methods from literature, and evaluate their effectiveness across different datasets, similarity scores between data points, expected distribution, and computational constraints. Furthermore, this article offers open-source code for the imple mentation of these methods.
BibTeX
@misc{costa2023reliable,
  author = {José Lucas De Melo Costa and Eliott Barbot},
  title = {{Reliable AI through Out-of-Distribution Detection: A Benchmark Study}},
  year = {2023},
  url = {https://openreview.net/forum?id=0Q9lmUuYGGs},
  howpublished = {OpenReview},
}

Loading related content...

Figure