Ambiguity representation and resolution in spoken dialogue systems

Egbert Ammicht, Alexandros Potamianos, Eric Fosler-Lussier

TL;DR

This paper introduces a novel, domain-independent approach to representing and resolving ambiguities in spoken dialogue systems. By separating raw data from derived candidates and employing a pragmatic scoring mechanism combined with explicit user correction, they significantly reduce errors in travel reservation dialogues.

What Problem Does This Paper Address?

Spoken dialogue systems constantly grapple with ambiguity. This can stem from various sources: the inherent vagueness of natural language (e.g., "next Saturday" could mean this week or next), errors from speech recognition systems (which are still quite prevalent), and even user corrections that might themselves be ambiguous. Effectively handling these ambiguities is crucial for a smooth and successful user interaction. The authors aim to create a system that can robustly understand and resolve these ambiguities in a way that is not tied to a specific application domain.

Methods

The core of their approach lies in a novel semantic representation and a multi-stage resolution strategy.

  • Prototype Tree: A domain-independent tree structure representing the ontology of a domain (e.g., a travel agent's knowledge about flights, hotels, and cars). This tree defines concepts and their relationships, serving as the backbone for semantic representation.
  • Notepad Tree: A dynamic tree that mirrors the prototype tree, used to store attribute-value (AV) pairs extracted from user utterances.
  • Application Tree: Stores derived candidate values for attributes, allowing for multiple possibilities to be considered simultaneously.
  • Context Tracking: Algorithms that match user input (partial attributes) with the current dialogue context to form complete attributes. This involves exact, interpolated, and overlap matches, with mechanisms to handle context changes.
  • Scoring Mechanism: Each candidate value is assigned a score (confidence factor) between -1 and 1. These scores are updated based on various evidence sources: raw data, match type from context tracking, and pragmatic analysis. A candidate is selected only if its score exceeds a threshold and is sufficiently higher than the next best candidate.
  • Pragmatic Analysis: The system analyzes user responses to prompts to infer confirmation or objections, adjusting candidate scores accordingly. This allows for implicit confirmation of data.
  • Correction Mechanisms: Users can directly address attributes with commands like "clear," "freeze," or "change," providing explicit control over the dialogue.

Datasets

The experimental results are based on an analysis of 35 dialogues collected during the June 2000 DARPA Communicator evaluation. These dialogues were run through both their previous system and the new system to compare performance. No new benchmarks were introduced.

Key Insights and Ideas

  • Domain Independence is Key: By abstracting domain knowledge into a parameterizable "prototype tree," the system's core ambiguity resolution mechanisms can be reused across different applications.
  • Separation of Concerns: The clear distinction between raw data (notepad tree) and derived possibilities (application tree) simplifies the development and application of scoring and selection algorithms.
  • Pragmatics as a Powerful Signal: Incorporating pragmatic analysis of user responses (beyond just direct answers) provides strong evidence for or against candidate values, significantly improving disambiguation.
  • User Control is Essential: Explicit correction mechanisms empower users to override system decisions and guide the dialogue, making the system more robust to errors and user intent.

Key Results

The paper reports that their new ambiguity resolution strategy significantly improves dialogue interaction. Specifically, it eliminated about half of the errors in dialogues from a travel reservation task. In an analysis of 49 dialogue turns where systems diverged, their current system showed improvement in 25 cases, compared to only 3 cases where the older system was superior. These improvements were attributed to better parsing, the introduction of scoring and pragmatic analysis, and the interaction of these modules.

Limitations and Future Work

The authors acknowledge that the system's performance relies on careful tuning of scoring functions and parameters. They also mention that not all domain knowledge can be encoded in the prototype tree, with some specialized knowledge requiring task-dependent procedures. Future work includes further tuning of scoring algorithms and integrating confidence measures from other system modules (like speech recognition).

Summary updated

Abstract

Spoken natural language often contains ambiguities that must be addressed by a spoken dialogue system. In this work, we present the internal semantic representation and resolution strategy of a dialogue system designed to understand ambiguous input. These mechanisms are domain independent; task-specific knowledge is represented in parameterizable data structures. Speech input is processed through the speech recognizer, parser, interpreter, context tracker, pragmatic analyzer and pragmatic scorer. The context tracker combines dialogue context and parser output to yield raw attribute-value (AV) pairs from which candidate values are derived. The pragmatic analyzer adjusts the confidence associated with each AV candidate based on system intent, e.g., implicit confirmation, and on user input. Pragmatic confidence scores are introduced to measure the dialogue managers confidence for each AV: MYCIN-like scoring is used to merge multiple information sources. Pragmatic analysis and scoring is combined with explicit error correction capabilities to achieve efficient ambiguity resolution. The proposed strategies greatly improve dialogue interaction, eliminating about half of the errors in dialogues from a travel reservation task.
BibTeX
@inproceedings{ammicht2001ambiguity,
  author = {Egbert Ammicht and Alexandros Potamianos and Eric Fosler-Lussier},
  title = {{Ambiguity representation and resolution in spoken dialogue systems}},
  booktitle = {Interspeech},
  year = {2001},
  doi = {10.21437/eurospeech.2001-523},
}
Figure