Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants

Cheng Qian, Haode Qi, Gengyu Wang, Ladislav Kunc, Saloni PotdarEMNLP 2022

TL;DR

This paper tackles the critical challenge of "Out-of-Scope" (OOS) detection in virtual assistants (VAs), aiming to accurately identify user queries that a VA cannot handle. They propose a simple yet effective modification to existing multi-stage OOS detection methods that jointly models intents, entities, and OOS classification, leading to improved performance and robustness in real-world enterprise settings.

What Problem Does This Paper Address?

Virtual assistants (VAs) are becoming increasingly sophisticated, but they still struggle with queries that fall outside their predefined capabilities. Accurately identifying these "Out-of-Scope" (OOS) queries is crucial for a good user experience. If a VA incorrectly tries to answer an OOS query, it can lead to frustration and a poor user experience. Conversely, if it frequently hands off queries that it could handle, it reduces efficiency and perceived intelligence. This problem is particularly challenging in enterprise VAs because:

  • Semantic Overlap: OOS queries often have a high semantic overlap with "In-Scope" (IS) intents, making them difficult to distinguish from legitimate queries. These are termed "In-Domain OOS" (ID-OOS).
  • Data Imbalance and Noise: Real-world data provided by VA designers is often imbalanced (some intents have many examples, others few) and noisy (typos, slang, emojis).
  • Entity Importance: Entities (like dates, names, product models) play a vital role in VA understanding, but academic OOS detection often overlooks their integration.
  • Designer Expertise: VA designers are not typically machine learning experts, requiring robust and easy-to-manage solutions.

Methods

The authors propose a modification to a multi-stage OOS detection approach. Instead of treating OOS detection as a purely binary classification problem, they integrate it with intent classification. Their core idea is to use the OOS detection score to "discount" the confidence score of the IS classifier.

Here's a breakdown:

  • Multi-stage OOS: This approach first uses a binary classifier to determine if a query is IS or OOS. If it's deemed IS, subsequent stages determine the specific intent.
  • Discounting Approach: Their novel contribution is how they use the OOS score. They calculate a "discounting factor" based on the OOS classifier's output (specifically, the cosine distance to the nearest neighbor in an embedding space). This factor is then used to adjust the IS classifier's confidence score. This allows for a more nuanced decision, where a query with high similarity to an IS intent but also flagged as potentially OOS by the OOS classifier will have its confidence reduced.
  • Joint Modeling: Crucially, their approach emphasizes modeling entities alongside intents and OOS classification. They preprocess entities and their synonyms before generating sentence embeddings, ensuring that entity information is considered.
  • One-Class Classification: For scenarios with no OOS training examples, their method can adapt to a one-class classification approach.

Datasets

The authors evaluated their method on a comprehensive benchmark comprising 8 publicly available datasets:

  • ATIS
  • BANKING77
  • CLINC150 (evaluated on full set, banking, and credit card subsets)
  • StackOverflow
  • SNIPS
  • HAR
  • ROSTD (including a "coarse" version and specific ID-OOS examples)
  • HINT3 (evaluated on SOFMattress, Powerplay11, and Curekart subsets)

They also note that for datasets lacking ID-OOS examples, they artificially created them by treating a percentage of IS intents as OOS.

Key Insights and Ideas

  • ID-OOS is the Real Challenge: The paper highlights that distinguishing "In-Domain OOS" (ID-OOS) queries, which are semantically similar to existing intents, is far more difficult than distinguishing "Out-of-Domain OOS" (OOD-OOS) queries.
  • Entities are Crucial: Ignoring entities in OOS detection leads to suboptimal performance. Their method's ability to incorporate entity information is a key differentiator.
  • Discounting Confidence is Key: Simply using a binary IS/OOS classifier or a multi-class approach can be brittle. Discounting the IS confidence based on OOS signals provides a more robust way to handle ambiguous queries.
  • Simplicity Aids Deployment: The proposed method is described as a "simple modification," which is vital for enterprise adoption where ease of deployment and maintenance is paramount.

Key Results

The proposed "Discounting (Our Approach)" consistently outperformed other methods across various metrics on their benchmark datasets.

  • Overall Accuracy: Achieved the highest overall accuracy, indicating better overall classification performance.
  • OOS Performance: Showed strong performance in OOS detection metrics like OOS F1 and OOS recall, suggesting it's better at correctly identifying OOS queries.
  • Robustness to ID-OOS: Performed well on ID-OOS examples, demonstrating its ability to handle semantically similar but out-of-scope queries.
  • Online Evaluation: During real-world deployment, they observed that the change in top confidence scores was minimal for over 85% of live traffic, indicating a smooth transition and minimal disruption to existing workflows.
  • Computational Efficiency: The OOS detection model size was kept under 150MB, and training time was within 30 seconds, making it suitable for large-scale enterprise VA platforms.

Limitations and Future Work

  • Language Limitations: The current work focuses on English. While they mention extending to European languages with similar gains, extensive benchmarking for other languages is out of scope.
  • Code Switching: The paper does not evaluate performance on code-switching (mixing languages within a single utterance), which is common in real-world deployments.
  • Synthetic OOS Data: The paper acknowledges that while synthetic OOS examples can be useful, they require caution due to the potential for introducing spurious correlations.
  • Future Work: The authors suggest further exploration of their approach with different OOS algorithms (they briefly show results with an autoencoder) and more extensive benchmarking.

Summary updated

BibTeX
@inproceedings{qian2022distinguish,
  author = {Cheng Qian and Haode Qi and Gengyu Wang and Ladislav Kunc and Saloni Potdar},
  title = {{Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
}
Figure