Context Management with Topics for Spoken Dialogue Systems

Kristiina Jokinen, Hideki Tanaka, Akio YokooACL 1998

TL;DR

This paper introduces a novel topic-based context management model for spoken dialogue systems. By structuring domain knowledge into a topic tree and using a "Predict-Support" algorithm, the system can more accurately predict and understand the user's intent, leading to more robust and accurate dialogue processing, even with noisy speech recognition.

What Problem Does This Paper Address?

Spoken dialogue systems (like voice assistants) struggle with errors from speech recognition, which can cascade and lead to misunderstandings. While previous approaches used "speech acts" (like "request" or "inform") to manage context, these are too abstract and ignore the semantic content of utterances. This paper addresses the need for a more robust context management system that can handle these errors by focusing on the information content of utterances, represented by topics within a domain. This is crucial for maintaining dialogue coherence and improving the overall performance of spoken dialogue systems.

Methods

The core of the paper is the proposed topic model and the Predict-Support algorithm.

  • Topic Model:
    • Topic Tree: Domain knowledge is structured hierarchically into a "topic tree." This tree represents expected topic shifts, guiding the system on what is likely to be discussed next.
  • Topic Vectors: Each word is assigned a "topic vector" that quantifies its mutual information with different topic types. This captures how much information a word provides about a specific topic.
  • Predict-Support Algorithm: This algorithm assigns topics to utterances in two main steps:
  • Prediction: It uses the topic tree and prior probabilities of topic shifts to predict likely next topics based on the dialogue history.
  • Support: It then uses the topic vectors of the words in the current utterance to calculate how well each predicted topic is supported by the actual words spoken.
  • The algorithm then selects the most supported topic, either by defaulting to the predicted topic, relying on the "what-is-said" heuristic (if supported topics don't match predictions), or the "what-is-talked-about" heuristic (if words don't support any topic).

Datasets

The experiments were conducted on the ATR Spoken Language Dialogue Database, which contains 80 dialogues from a hotel reservation and tourist information domain. This corpus includes approximately 4228 utterances and is tagged with both speech acts and topic types.

Key Insights and Ideas

  • Topic Information is Key: The paper strongly argues that focusing on topic information, which captures the semantic content of utterances, is more effective for robust context management than abstract speech acts.
  • Hierarchical Structure Aids Prediction: The topic tree provides a valuable top-down constraint, guiding the system's predictions and making the context management more efficient and coherent.
  • Mutual Information for Word-Topic Links: Using mutual information to define word-topic relationships is a sophisticated way to capture the nuanced support a word can offer to different topics.
  • Predict-Support Balances Top-Down and Bottom-Up: The algorithm effectively combines top-down predictions from the topic tree with bottom-up evidence from the actual words spoken, creating a robust system.
  • Handling Noisy Input: The model is designed to improve word recognition accuracy by providing contextual expectations, and it aims to compensate for errors in the speech recognition output.

Key Results

The Predict-Support algorithm achieved a first-ranked prediction accuracy of 78.68% and a precision of 74.64% on the pruned topic types. This is presented as a significant improvement over speech act recognition rates reported in previous work, especially when considering the perplexity of the corpus. The paper also notes that a finer set of topic tags (62 vs. 10) can worsen accuracy, but the Support-part of the algorithm effectively remedies prediction inaccuracies.

Limitations and Future Work

  • Data Sparsity: The authors acknowledge the problem of sparse data when calculating mutual information. This can lead to issues with out-of-domain words and words that only appear with a single topic type in the training data.
  • Manual Topic Tree Creation: The topic trees used in the experiments were hand-coded, which is time-consuming and subjective. An automatic clustering program for building topic trees is under development.
  • Future Work:
    • Developing automatic methods for domain modeling and topic tree construction.
  • Generalizing topic trees to incorporate world knowledge and conceptual relations.
  • Further investigating the relationship between topics and speech acts to define their respective roles in context management.
  • Addressing inaccuracies in calculating mutual information due to sparse data.

Summary updated

BibTeX
@inproceedings{jokinen1998context,
  author = {Kristiina Jokinen and Hideki Tanaka and Akio Yokoo},
  title = {{Context Management with Topics for Spoken Dialogue Systems}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1998},
}
Figure