TL;DR
This paper introduces an utterance-level confidence scoring system for conversational AI that aims to reject misunderstood user inputs rather than responding incorrectly. By combining recognition-based features with semantic and linguistic cues, they achieve over 60% correct rejection while maintaining a 98% acceptance rate for correctly understood utterances.
What Problem Does This Paper Address?
Conversational AI systems, especially those deployed for wider public use, often struggle with understanding spontaneous, noisy, or out-of-domain user utterances. When a system misunderstands, it can lead to frustrating user experiences by providing incorrect responses. Traditional approaches often rely on a simple "top N" hypothesis rejection, which can be insufficient. This paper addresses the need for a more sophisticated confidence scoring mechanism that can reliably distinguish between correctly understood and misunderstood utterances at the utterance level, thereby improving system robustness and user satisfaction.
Methods
The authors propose an utterance-level confidence scoring system that leverages a combination of features. A key innovation is their automatic annotation algorithm for creating training data. Instead of manual labeling, they parse both the reference transcription and the recognizer's hypotheses to generate semantic frames. An utterance is considered correctly understood if the semantic frame from a hypothesis matches the frame from the reference.
For confidence scoring, they explore two main categories of features:
- Recognition-based Features: These include standard metrics like N-gram language model scores, acoustic scores, and the number of hypotheses. They also consider features derived from the structure of N-best hypotheses.
- Linguistic and Application-Specific Features: This is where the novelty lies. They incorporate features related to:
- Parse Quality: Whether a hypothesis can be fully or partially parsed, and the parse probability.
- Semantic Features: Assigning weights to different word classes (e.g., geography, content words) and calculating semantic weights of utterances and semantic distances between hypotheses. This allows the system to understand the importance of specific words and the degree of difference in meaning between potential interpretations.
They use Fisher Discriminant Analysis (LDA) iteratively to select the optimal set of features for classification. They also explore Decision Trees to understand feature interactions and provide more interpretable rejection cues.
Datasets
The experiments were conducted using data collected from users interacting with the JUPITER weather information system. This is a telephone-based conversational system.
- Training Data: A subset of the collected utterances.
- Development and Test Sets: Three independent sets of data were used for rejection experiments.
- Final Testing: Performed on yet another independent test set.
The paper notes that over 59,000 utterances from over 10,560 callers were collected and orthographically transcribed. A baseline recognizer achieved a 15% word error rate on an independent test set.
Key Insights and Ideas
- Utterance-level features are sufficient: The authors demonstrate that focusing on utterance-level features, rather than just word-level confidence, can effectively score the confidence of an entire utterance.
- Semantic matching is powerful: Comparing semantic frames generated by a parser proves to be a robust method for automatically labeling training data, achieving high agreement with manual annotations. This highlights the importance of understanding meaning for confidence scoring.
- Semantic importance matters: Assigning weights to different word classes and calculating semantic distances between hypotheses allows the system to better identify utterances with potentially significant meaning differences, which are more likely to be misunderstood.
- User behavior is influenced by rejection: The study observed that users are more likely to hang up after an incorrect rejection than after multiple rejections, suggesting that the system's feedback mechanism is crucial for user retention.
Key Results
The proposed confidence scoring system, using features selected by Fisher LDA, achieved:
- 60% correct rejection rate on the development and test sets.
- 98% acceptance rate for correctly understood utterances.
- A false rejection rate of 3% on the test set.
The automatic annotation process agreed with manual annotation in 90.6% of cases, and was estimated to be 98.1% accurate. The overall understanding rate of the system (based on orthographic transcriptions) was around 57.2%, reflecting the challenges of spontaneous speech.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the overall understanding rate of the system (57.2%) is lower than ideal, particularly for spontaneous speech and out-of-domain queries.
- The current rejection message ("Sorry, I'm not sure what you said") is not very helpful and can lead to subsequent rejections.
- While decision trees helped identify useful features, they showed greater performance degradation from development to test sets compared to Fisher LDA.
- Future Work:
- More informative feedback: The authors plan to provide more specific feedback to users, such as suggesting alternative interpretations for content words (e.g., "Did you say Boston, Massachusetts or Austin, Texas?").
- Word-level confidence: Integrating word-level confidence measures to detect early problems with specific content words.
- Leveraging decision trees: Using the decision tree structure for improved user feedback mechanisms.
Summary updated