Abstract
This paper introduces a framework that dynamically updates speech recognition word probabilities by tracking a user’s goals through a Bayesian Network (BN). By leveraging information from previous turns in a conversation, the system can better predict specific content words, significantly improving the precision of concept understanding in spoken dialog systems. Evaluation on the CMU Let’s Go corpus demonstrates that while Word Error Rate (WER) remains stable, the precision of extracted concepts improves by over 5% absolute, approaching oracle performance.
Introduction
In Spoken Dialog Systems (SDS), Automatic Speech Recognition (ASR) typically relies on Statistical Language Models (SLMs). Most existing systems use "state-dependent" models—meaning the system knows that if it just asked "Where are you leaving from?", the user is likely to provide a location.
However, these models are often "blind" to the specific content of the conversation history. If a user mentioned earlier that they are heading to the airport, a standard system doesn't use that knowledge to increase the probability of the word "airport" in subsequent turns. The authors tackle this gap: how can a system use the evolving context of a user's specific goals to improve the accuracy of speech understanding?
Methodology
The authors propose a Dynamic Language Modeling framework that combines three contextual factors: linguistic variability (n-grams), discourse structure (state-dependent prompts), and user goals. The novel addition is the use of a Bayesian Network (BN) to model the user's intent as a set of interconnected random variables.
As the dialog progresses, the system extracts concept-value pairs from the ASR hypothesis. These are fed into the BN as evidence, which then updates the "belief" (marginal probabilities) for all related concepts. These updated probabilities are then "plugged back" into a class-based SLM. For example, if the BN determines a high probability that the user is traveling to "downtown," that specific word receives a significant probability boost in the language model for the next turn. The researchers tested two versions: NoConfirm (updates the BN immediately based on the top ASR result) and Confirm (only updates after the user explicitly confirms a value).
Experimental Setup
The approach was evaluated using the CMU Let’s Go Corpus, a publicly available dataset of telephone-based bus information dialogs. The training set consisted of 4,010 dialogs (over 48,000 utterances), while the test set included 208 dialogs (1,931 utterances). The domain involves six core concepts: Bus Route, Departure/Arrival Neighborhood, Departure/Arrival Stop, and Travel Time.
Key Results and Discussion
The researchers compared their BN-based approach against a baseline state-dependent SLM and an "Oracle" (which has perfect knowledge of the user's goal).
A primary finding was the Precision Boost: the "Confirm" system improved concept understanding precision by 5.2% absolute over the baseline, nearly reaching the performance of the Oracle (64.5%). Interestingly, the overall Word Error Rate (WER) remained almost identical. This is because content words (like specific bus stops) make up only a small fraction of the total words spoken, even though they are the most critical for the system's success.
Furthermore, the model provides significant utility by penalizing inconsistent values. It is often easier for the system to predict that a user won't say a specific stop (because it's not on the current route) than to perfectly predict exactly which stop they will say. This reduces "false positives" in understanding.
Conclusion and Future Work
While the dynamic language model improves precision, it remains sensitive to ASR errors. In the "NoConfirm" model, misheard evidence can bias the BN toward the wrong goal, making recovery difficult. Future work suggests incorporating confidence scores and n-best lists from the ASR to handle uncertainty more gracefully. Additionally, while the Let’s Go domain is relatively simple, the authors believe the benefits of this approach would be even more pronounced in richer contexts, such as embodied agents or complex multi-step planning tasks.
Summary updated